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ABSTRACT 


The Combined Arms Analysis Tool for the 21st Century, or COMBATXXI, is 
the primary analytical combat simulation model in use by the Marine Corps’ Oper¬ 
ations Analysis Division (OAD) and the Army’s Training and Doctrine (TRADOC) 
Analysis Center for weapon system and force effectiveness analysis. 

The bottleneck in the COMBATXXI scenario production process is the behav¬ 
ior development process. Analytically useful scenarios demand complex and dynamic 
behaviors that react to the unique circumstances of the simulation’s current state. 
Hierarchical Task Networks (HTN) are the state-of-the-art methodology in COM¬ 
BATXXI used to describe dynamic behaviors. Although HTNs decrease the scenario 
development time, they are difficult to conceptualize, validate, and troubleshoot. 
The long iteration cycle is due, in part, to the complex development environment, 
the necessity of a large simulated infrastructure to test behaviors, and an inability to 
visually debug. 

Here we present a solution for prototyping HTNs by extending an existing 
commercial implementation of Behavior Trees within the Unity3D game engine prior 
to building the HTN in COMBATXXI. Existing HTNs were emulated within this 
prototyping environment to test transferability of the behaviors, and new HTNs were 
prototyped in Unity3d prior to being built in COMBATXXI as a proof of concept. 
Prototyping HTNs in a 3D development environment may prove useful by reducing 
the iteration time and improving the overall quality of the behaviors. The interactive 
nature of Unity3d reduces the iteration time, and the ability to rapidly test many 
different cases improves the quality of the behaviors. 
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CHAPTER 1: 

Improving the Behavior Development Process 


The Combined Arms Analysis Tool for the 21st Century (COMBATXXI) is the pri¬ 
mary multiagent-based simulation tool in use by the Marine Corps’ Operations Anal¬ 
ysis Division (OAD), under the purview of the Deputy Commandant for Combat 
Development and Integration (CD&I), as well as the Army’s Training and Doctrine 
Command (TRADOC) Analysis Center. Analytically useful scenarios demand com¬ 
plex and dynamic behaviors which allow agents to react to the unique circumstances 
of the simulation’s current state. Hierarchical Task Networks (HTN) are the state-of- 
the-art methodology in COMBATXXI used to describe dynamic behaviors. Although 
HTNs greatly decrease the scenario development time, they are difficult to concep¬ 
tualize, validate, and troubleshoot. The long iteration cycle is due, in part, to the 
complex development environment, the necessity of a large simulated infrastructure 
to test behaviors, and an inability to visually debug. 

In this thesis, we suggest a construct for prototyping HTNs by extending an existing 
commercial implementation of Behavior Trees within the Unity3D game engine prior 
to building the HTN in COMBATXXI. Prototyping HTNs in a 3D development 
environment may prove useful by reducing the iteration time and improving the overall 
quality of the behaviors. The iteration time is reduced due to the interactive nature 
of Unity3D and the availability of low-cost models. The quality of the behaviors is 
improved due to the ability to rapidly test many different cases. 

This chapter provides a historical and contextual overview of agent-based simulations 
in the Marine Corps analysis community, and a look ahead at the chapters to follow. 1 


1 We recognize that varying definitions of "agent-based" simulation exist in the Modeling and 
Simulation (M&S) community. Our interpretation is that of a simulation which revolves around 
sensing and reacting agents, which some have called "entity-level combat simulations" [1, p. 608]. 
These models "represent individual combat vehicles (e.g. tanks or helicopters), platforms (e.g. radar 
sets), and personnel (e.g. infantry squads or soldiers) as distinct simulation entities. Necessary 
state information, such as location, velocities, and ammunition load, is maintained for that entity 
separately and each entity is capable of independent action." [1] 
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1.1 Agent-based Simulations in the Marine Corps 

Agent-based simulations provide a framework in which analysis can be conducted 
without the limiting assumptions made in many conventional analytic models. In 
recent decades, the computational power available to simulation developers has in¬ 
creased many times over, and simulations are becoming a primary analytic tool rather 
than "a method of last resort" [2, p. 294], A primary the benefit of multi-agent based 
simulations of combat is the agent-based behaviors, regardless of whether they are 
explicitly scripted or dynamic in nature. 

The Marine Corps has embraced the analytic potential of agent-based simulations 
since their infancy, most famously in the study titled Complexity & Combat , spon¬ 
sored by Lieutenant General Van Riper [3]. Currently, this banner is carried by the 
Marine Corps’ Operations Analysis Division (OAD), within the Analysis Directorate 
of the Deputy Commandant, Combat Development and Integration (CD&I). The 
primary mission of OAD is to provide analytic support to decision makers “about 
weapon systems, equipment acquisition and resource allocation” [4], One way in 
which this is achieved is through the use of agent-based combat simulations, such 
as the COMBATXXI. COMBATXXI is a “high-resolution, closed-form, stochastic, 
discrete event, entity level structure analytic combat simulation”, which focuses on 
land-based combat with some littoral and amphibious operations support [5]. 

COMBATXXI is not the first agent-based simulation that the Department of De¬ 
fense (DoD) has used to support its analysis. 2,3 However, COMBATXXI and its 
predecessors alike have been guided by the Marine Corps’ philosophy on combat and 
warfare at large. The primary bearer of the high-level philosophy on warfare for 
the Marine Corps is the Marine Corps Doctrinal Publication 1 (MCDP-1), titled 
Warfighting, which states: 

"War is not governed by the actions or decisions of a single individual in any one place 
but emerges from the collective behavior of all the individual parts in the system 

2 The Combined Arms and Support Task Force Evaluation Model (CASTFOREM), for example, 
is the predecessor to COMBATXXI. Details of CASTFOREM can be found in [3, p. 42] and here: 
http://www.simscript.com/solutions/military/CASTFOREM.html 

3 An overview of previously existing combat simulations can be found in section 1.4 of Ilachinski’s 
Artificial War [3, p. 39]. 
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interacting locally in response to local conditions and incomplete information. A 
military action is not the monolithic execution of a single decision by a single entity 
but necessarily involves near-countless independent but interrelated decisions and 
actions being taken simultaneously throughout the organization." [6] 

The Marine Corps defines war as a violent clash of wills, wherein each opponent is 
not a single or homogeneous entity, but a collection of independent actors [6]. It is 
this philosophy of war that drives the Marine Corps’ outlook on analytic simulations, 
and makes its analysis community ever more inclined to the use of multi-agent based 
simulations. It is with this philosophy in mind that this thesis aims to improve upon 
the behavior-based methods which make those simulations useful. 

This chapter provides a brief historical overview of the emergence of dynamic behav¬ 
iors within analytic simulations used by the United States Marine Corps (USMC) 
and the underlying motivation for further research in this area. 


1.2 Complexity Theory in Simulations for Analysis 

In Dr. Andrew Ilachinski’s book titled Artificial War: Multiagent-Based Simulation of 
Combat, he details the history behind how the Marine Corps came to use multiagent- 
based simulations. From 1996-2003, the Center for Naval Analysis (CNA) directed a 
project called Complexity & Combat which was sponsored by the Commanding Gen¬ 
eral of Marine Corps Combat Development Center (MCCDC), Lieutenant General 
Van Riper. The primary goal of the project was to determine the “general applica¬ 
bility of nonlinear dynamics and complex systems theory to land warfare.” Thus the 
Marine Corps’ relationship with complexity theory was born. Although the Marine 
Corps’ philosophy of war is quite similar to complexity theory, no organizational ef¬ 
forts had been made prior to the CNA project to implement complexity theory within 
models of warfare. This advancement towards using complexity theory within models 
of warfare as a peer or replacement for traditional, attrition-based models, represents 
the aggressively revolutionary mindset of Lieutenant General Van Riper for which he 
is so well known. 
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1.2.1 Complexity Theory, Defined 

Complexity theory, also known as complex systems theory, is the idea that "compli¬ 
cated systems can generate simple behavior." [3, p. 5] This concept applies to the 
study of complex organisms, including humans. According to Dr. Ilachinski, research 
in the area of complex systems theory can be summarized by two themes: 

Complexity emerges from simplicity. This is the concept that relatively few and 
simple behaviors have the ability to result in seemingly complex individual 
or group behaviors. Perhaps the most famous example of this is the Boids 
simulation, first created by Craig Reynolds in 1986, in which complex flocking 
behaviors emerge as a result of three simple parameters: separation, alignment, 
and cohesion [7]. 

Simplicity emerges from complexity. This is the concept that relatively stable 
results arise from an otherwise complex model. This may occur through so- 
called “self-organization” of agents within the simulation, or through guided 
tuning by the scenario or behavior designer. An example may be a behavior at 
the appropriate echelon of leadership level which dictates that a unit reorganizes 
in a smaller perimeter as it takes casualties, in order to fill the gaps of its defense. 
This is the portion of complexity theory that agent-based simulations rely on, 
and this is the portion of complexity upon which this thesis is focused [3, p. 5]. 

It should be noted that agent-based simulations are only one tool which aims to 
answer research questions in the area of complexity theory. Readers are directed to 
Dr. Ilaehinski’s book for a more comprehensive list of tools that are used in this 
domain [3, p. 6]. 


1.3 Analytic use of COMBATXXI 

COMBATXXI is the simulation tool currently used and jointly developed by the 
Marine Corps’ OAD and the Army’s TRADOC Analysis Center. COMBATXXI has 
many of the same goals as Dr. Ilaehinski’s original combat simulation tool, known as 
Irreducible Semi-Autonomous Adaptive Combat (ISAAC). Both tools aim to leverage 
the insights gained from complexity theory as it applies to military simulations for 
analysis. COMBATXXI is capable of modeling an entire Marine Air Ground Task 
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Force (MAGTF) of various sizes, including manned and unmanned aviation assets, 
complex land-based combat operations, and amphibious operations. There is support 
for simulating urban and asymmetric combat as well as more conventional combat 
operations. It provides simulationists with a wide range of options for scenario devel¬ 
opment, and additional modules can be developed to suit the requirements of specific 
studies. COMBATXXI is the focus of this thesis because it is the primary simulation 
used by OAD. All succeeding discussions regarding "simulations" and "models" can 
be assumed to pertain most specifically to COMBATXXI. 


1.3.1 The Role of Behaviors in Agent-Based Simulations 

Ilachinski’s latter theme of complexity theory states that simplicity has the potential 
to emerge from complexity. The clarity of analysis which we seek from multi-agent 
based simulations of combat aims to leverage this theme. Due to the closed-loop na¬ 
ture of COMBATXXI, simulationists must have the ability to manually script behav¬ 
iors or build dynamic behaviors which can determine appropriate actions at runtime. 4 
These behaviors must sufficiently emulate doctrinal military behaviors in order to be 
valid. In practice, current methodologies embody both scripted and dynamic behav¬ 
iors. For example, the observation process in COMBATXXI is dynamic in that it is 
always running, once activated, and requires no further explicit commands from the 
simulation developer. On the other hand, maneuver is typically scripted in a manual 
fashion by stringing together way-points into routes which the agents explicitly follow. 

Maneuver-related behaviors may also be dynamic. For example, a dynamic behavior 
may measure which route is the fastest and provides the most cover from potential 
enemy observations and fields of fire. These behaviors, be they dynamic or scripted, 
are a primary enabler for multi-agent based simulations of combat. This thesis exam¬ 
ines ways in which the development methods for the latter type of scenario-specific 
dynamic behavior may be improved upon. 


4 Scripted behaviors are those which do not change based on varying stimuli, while dynamic 
behaviors will adjust based on different input according to their programmed algorithm. 
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1.4 The Scenario Design Bottleneck 

Manual scripting of behaviors, such as those that dictate maneuver routes, is a tedious 
process that gets more complicated whenever its scope is expanded in space, time, 
or number of agents. While there are many potential bottlenecks in the design of 
COMBATXXI scenarios, the primary limiting factor is the development of scripted 
agent-based behaviors which adequately emulate military doctrine and robustly apply 
that doctrine to a wide variety of situations. 


1.4.1 Refactoring without Dynamic Behaviors 

Refactoring scenarios in COMBATXXI without the use of behaviors which dynam¬ 
ically adjust to the new scenario can be excessively wasteful. Even small changes 
can have drastic ripple effects throughout the simulation, leading to adversely large 
developer hour investments for seemingly trivial changes, particularly later in the de¬ 
velopment process. For example, if a COMBATXXI scenario is being built to support 
an analysis of alternatives (AoA), the scenario designer may be tasked with swapping 
out different types of a certain armored vehicle. Perhaps one variant has a longer 
range primary weapon system, and therefore would take a different route in open 
terrain in order to leverage this advantage. The scenario designer would have to 
build a new set of routes for each armored vehicle in the scenario. This is where the 
advantage of dynamic behaviors lies. 

Dynamic behaviors typically have parameterized input which allows for ease of refac¬ 
toring. For example, consider a formation which has been manually built to include 
5 meters of dispersion for a simulation scenario in open terrain. If the scenario man¬ 
ager wants to examine how the outcome changes based on dispersion, several different 
formations would have to be manually built. A dynamically programmed formation 
may include a parameter for dispersion, allowing for that same formation to be useful 
in many different scenarios and to be easily changed within a scenario. Refactoring 
scenarios becomes significantly easier with dynamic behaviors. 
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1.4.2 COMBATXXI Dynamic Behavior Development 

Despite the significant positive benefits that can be achieved from the use of dynamic 
behaviors within COMBATXXI, they are often difficult to conceptualize, validate, 
and troubleshoot. The long iteration cycle is due, in part, to the complexity of the 
development environment, the necessity of a large simulated infrastructure to test 
behaviors, and an inability to visually debug in real-time. A detailed discussion of 
dynamic behavior development and testing in COMBATXXI will follow in chapter 3. 


1.5 Commercial Game Development Best Practices 

Many practices from the commercial game industry have the potential to benefit the 
development of agent-based behaviors for simulations. This work draws inspiration 
from those practices for a variety of reasons. For example, both domains reward soft¬ 
ware optimization, code re-use, and the ability to quickly refactor or extend existing 
code. The goals of game design may be to maximize a customer’s interest while min¬ 
imizing the computational cost of running the game on a variety of platforms [8, p. 
155]. Similarly, simulation software must be computationally tractable, albeit for a 
different purpose. Instead of entertaining customers, simulationists aim to provide 
output data that is ready for consumption by military and civilian analysts. In or¬ 
der for this output to be statistically useful, it must be sufficient in quantity, so a 
simulation which takes too long to run will be of no use to anyone. These are just a 
few examples. Chapter 4 provides an overview of some best practices from the com¬ 
mercial game development industry which may prove beneficial to the development 
of dynamic behaviors for COMBATXXI. 


1.5.1 The Impact of the Behavior Development Environment 

The development environment is perhaps the most important tool for a scenario 
designer who is tasked with the development of dynamic behaviors. Due to the 
complex nature of these behaviors and the stimuli to which they respond, intuition 
and judgment alone are not sufficient to predict the resulting emergent behavior for a 
wide variety of use cases. User-friendly development environments allow designers to 
fit in more tests in a shorter amount of time, leading to more robust, stable behaviors. 
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The development environment should allow behavior designers to test their behaviors 
early and often, in order to prevent incorrect or unstable behaviors from creating 
complex bugs later in their life cycle [9, p. 237]. Section 4.2.1 details the qualities 
which the ideal behavior development environment should possess, as revised from 
previous work generally defining high-quality behaviors [10, p. 8]. 

1.5.2 Benefits of 3D Development Environments 

While the attributes specified in section 4.2.1 are generic, a specific attribute of an 
efficient behavior development environment is the inclusion of a 3D model viewer. 
A modern, interactive 3D model viewer leverages the "kinetic depth effect", which 
conveys 3D information to a user through a 2D interface-the computer screen [11], 
It is not only intuitive, but has been shown in user studies that an interactive 3D 
environment allows users to better understand spatial relationships between objects 
[ 12 ]- 

The effect of a 3D model viewer on the resulting quality of behavior development 
has not been studied in depth. However, we submit that the demonstrated ability 
of subjects to understand models better with 3D information translates to a better 
understanding of a model or agent’s behavior in a simulation. This then leads to 
a more thorough understanding of the implications of details within the behavioral 
logic, and therefore a better development process. 


1.6 Unity 3D as a Development Environment 

Game engines are growing in popularity due to their accessibility, affordability, and 
ease of authoring. There are large online communities which provide guidance, starter 
code, and support to anyone with a basic understanding of programming and the 
willingness to spend some time learning the nuance of that particular game engine. 
Unity3D in particular is free to use for small projects, and does not even require the 
payment of royalties for small productions. 5 Furthermore, the Unity Asset Store 
provides access to a plethora of low-cost models and development tools which help 
build artificially intelligent behaviors. 

5 Further details on the business model used by Unity can be found here: 
https: / / unity3d.com/legal / eula 



Unity3d was selected for purposes of this thesis due in part to its accessibility and 
the resident expertise within the Naval Postgraduate School. However, there may be 
other game engines which are also worth exploring. This thesis is a proof of concept, 
and does not serve as a singular solution for development of dynamic behaviors. For 
purposes of conciseness, Unity3D will be referred to as Unity. 

1.6.1 Behavior Designer 

The tool selected for the development of Behavior Tree (BT)s in Unity is Behavior 
Designer, version 1.5.5. Behavior Designer is a commercial off the shelf tool built by 
a small group of developers and sold in the Unity Asset Store. It is a BT authoring 
tool that ships with a large set of task nodes which can be used to build relatively 
complex behaviors without building any new tasks. It is authored in C#, which is 
the most common form of scripting in Unity. 

Other tools, such as Rain AI, were examined, but Behavior Designer was determined 
to be the most suitable for several reasons. The primary reason is that the source code 
is available upon purchase-enabling behavior designers to fully understand, debug, 
and extend the shipped software. Behavior Designer also provides an authoring tool 
which eases the authoring of advanced behaviors from the modular compilation of 
simple behaviors. 


1.7 Benefits of This Thesis 

From a practical perspective, this thesis explores a specific methodology for develop¬ 
ing a specific type of artificial intelligence for agents in COMBATXXI. It seeks to 
answer, in part, the Naval Research Program’s Broad Area Study NPS-N16-M159, 
which urges researchers to "examine the gaming industry for innovations below the 
individual application level that are applicable to the DoD modeling and simulation 
community." This thesis focuses on the second potential research question, which asks 
"What technology advances does the commercial game industry implement that can 
be leveraged by USMC modeling and simulation programs of record?" 

Ultimately, the primary goal of this thesis is to explore the benefit to agent behavior 
designers in using a 3D prototyping environment for the development of their agents’ 
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behaviors. This is demonstrated using Unity3D as a prototyping environment for 
behaviors that will be ported for use in COMBATXXI. 
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CHAPTER 2: 

Achieving Dynamic Agent Behavior 


Artificially intelligent, or dynamic behavior systems fall into two major categories: 
planners, or reactive. Planners are used as a means of abstracting the deliberate 
human planning process prior to an agent taking action within a simulation, or game, 
and looks into the future to do so [13, p. 1]. Planners take into account the world 
state and the methods with which they can change the world state in order to achieve 
a goal. The process of determining which methods to use to achieve the goal is known, 
in a formal sense, as planning. Often the plan is represented as a stack of tasks, or 
methods, to use in order to achieve the higher level goal. Contrastingly, reactive AI 
systems do not plan ahead, and only choose the most appropriate action given the 
current world state. 

These two broad categories encompass many different specific types of Artificial 
Intelligence (AI) systems, of which there are too many to list here. Their imple¬ 
mentations vary depending on the application domain, and the AI Game Develop¬ 
ment website provides a good overview of those used in the commercial game in¬ 
dustry [14]. This chapter examines only a few AI methods that are either currently 
used in COMBATXXI or could be useful for emulation of techniques employed within 
COMBATXXI. 

2.1 Hierarchical Task Networks 

Hierarchical Task Networks (HTN)s decompose a high-level task into a set of sub¬ 
tasks until no further decomposition is needed. This decomposition results in a stack 
of primitive tasks which, when executed, is expected to achieve the high-level goal of 
the HTN. Their popularity is due, in part, to their task decomposition method of 
planning, which is similar to human planning. 

HTNs, as they are typically described in academic sources, assume that the state 
of the world will not change in a way that will impact the plan unless changed by 
the agent [13, p. 229]. This allows for the entire plan to be created, from start to 
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finish, without concern of a condition which enables a primitive task later becoming 
invalid. This basic assumption is likely to cause problems in a computer simulation 
or game in which many agents are affecting the world state during the execution of 
a plan. Therefore, HTNs as implemented in commercial video games and analytic 
simulations tend to include a way to allow the agent to replan to avoid this problem. 
This methodology was first introduced to simulations for analysis by work conducted 
at the Naval Postgraduate School (NPS) [15]. 


2.1.1 HTNs in Video Games 

HTNs have become somewhat commonplace in video games over the last ten years as 
a means of achieving more realistic Al. The first major title to implement HTNs was 
Killzone 2 in 2009, although Stanford Research Institute Problem Solver (STRIPS)- 
like planners were introduced as early as 2005 with F.E.A.R. Planners were found 
to be more efficient to manage than reactionary AI mechanisms, such as finite state 
machine (FSM)s, due to their modularity and readability. Even large HTNs are easy 
to read, and small changes don’t have the wide-reaching effects on the behavior like 
they have a tendency to do with large FSMs [16]. 

HTNs are only one type of planner which have made their way into the commercial 
game development industry from academia and elsewhere. The other major type of 
planner, developed at Stanford in 1971, is called STRIPS [17]. Planners in games 
often deviate from the academic definition, but most can be categorized as either 
HTN-based or STRIPS-based [14], 

2.1.2 HTNs in COMBATXXI 

Traditional HTNs assume that the world state will only be changed by the planning 
agent. A planning system that only accounts for the agent’s actions is unlikely to 
be useful in the context of complex agent-based simulations. This is a well-known 
shortcoming of classic planning architectures in general. Ghallab et al. state that 
"a more realistic model interleaves planning and acting, with plan supervision, plan 
revision, and replanning mechanisms" [13, p. 9]. This is precisely the intent behind 
the replan triggers in the HTN implementation in COMBATXXI. The term HTN 
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will be used from this point forward to describe HTNs as they are implemented in 
COMBATXXI, unless otherwise noted. 

2.1.3 Anatomy of an HTN 

HTNs are represented as directed acyclic graphs composed of five types of nodes 
representing the following concepts: compound tasks, constraints, primitive tasks, 
goal tasks, and interrupt goal tasks [15], [18]. These fives task types are depicted in 
figure 2.1. 


Compound 

Task 


Figure 2.1: Five node types used in FITNs. Adapted from [15]. 






Compound Task 

Compound task nodes contain no task-oriented code and are used for syntactic struc¬ 
ture only. They may contain code which gathers information for use in following, sub¬ 
ordinate nodes, but this is not necessary. Compound task nodes represent a higher- 
level task which needs to be broken down into primitive tasks for execution [15, p. 
2 ], 


Constraint 

A constraint node directs the flow of execution within the HTN by the use of condi¬ 
tional logic. These nodes execute some code which resolves to a Boolean. The child 
nodes of the constraint node are executed only if the Boolean resolves to a value of 
true. 


Primitive Task 

Primitive tasks contain code representing some action, and require no further decom¬ 
position. Primitive tasks are added to the agent’s task stack to be scheduled as any 
other event in COMBATXXI. 
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Goal Task 

Goal tasks are a type of primitive task which, when finished, represent the goal which 
the HTN has attained. They “correspond to the achievement of the desired world 
state,” for which the HTN was built [15]. The execution of the HTN ceases after 
completion of the goal task. 


Interrupt Goal Task 

Interrupt goal tasks are specialized goal tasks which contain actions that are expected 
to take some time to execute, and will likely change the world state in some way that 
may require the HTN to be re-evaluated. The execution of the HTN is stopped until 
the actions in the interrupt goal task is completed. This prevents the entire HTN 
from being evaluated when it can be reasonably assumed that the conditions will 
change before the rest of the HTN is executed. This method is necessary for the 
scalability of HTNs within combat simulations, such as COMBATXXI, which may 
have thousands of agents using HTNs at the same time [15, p. 3]. 


2.1.4 Replanning with HTNs 

COMBATXXI is an event-driven simulation, and the events provide possible points at 
which the HTN planner could be run again. Due to the high number of agents whose 
behavioral processes need to be modeled in a combat simulation, replanning at each 
event would be computationally wasteful [15, p. 3]. Furthermore, high-frequency 
replanning could lead to the same sort of oscillating bugs that are a weakness of 
reactionary AI systems [19]. However, not replanning at all, like academic HTNs, 
would produce unsuccessful plans and unrealistic behaviors. Deciding what middle 
ground to achieve when implementing HTNs is clearly an important factor to their 
success. 

A primary benefit of the implementation of HTNs in COMBATXXI is their ability to 
replan based only on events which will affect the plan. This is accomplished by the 
HTN designer identifying "replan events" which the agent will listen for, and replan 
only when those events occur. Replan events are typically used in conjunction with 
"interrupt goal tasks," which pause the planner until execution up to that point has 
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finished. While "interrupt goal tasks" pause the tree to prevent planning too far into 
the future, they also identify points at which replanning will likely be necessary. 

2.1.5 GoToStore: An Example HTN 

As an explanatory example, we present an application agnostic scenario adjusted 
from [15], in which an agent is attempting to move from his home to the store. The 
scenario consists of three locations: the home, the store, and all points in between, 
and two objects: a car and a set of keys. The agent is at one of the locations, and 
either has the keys or not. The HTN representing this scenario is displayed in Figure 
2 . 2 . 



Figure 2.2: An HTN to get to the store. 


In this scenario, the agent first checks if he is already at the store. If so, then the goal 
node “Arrive at Store” is executed, and the HTN is complete. Otherwise, the agent 
checks to see if he is at home. If he isn’t, he walks to the store, thereby executing 
an interrupt goal task which pauses the rest of the tree until completion of this task. 
Upon completion of the “Walk to Store” interrupt goal task, the agent re-evaluates 
from the leftmost node, realizes he is at the store, and the HTN is again complete. 
If the agent started at home, he checks to see if he has the keys to the car. If so, he 
gets in the car, and drives to the store. Like the “Walk to Store” node, the “Drive to 
Store” node is also a goal interrupt task, which pauses the tree until its completion. 
At this point, the agent is at the store, and the HTN is complete. 
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The “Go To Store” HTN is built to achieve the high-level goal of getting to the store, 
and can stand alone in its current form. However, it could also be included within a 
more complex HTN as an interrupt goal task node, perhaps when the agent is out of 
bread and is trying to make a sandwich. An example is shown in Figure 2.3. 



Figure 2.3: An HTN to make a sandwich. 


It is in this way that the modularity of HTNs can be leveraged to create increasingly 
complex behaviors. Yet if we need to change the logic of the “Go To Store” portion of 
the behavior, we only need to change its HTN. This is a maintainability advantage 
over FSMs, and part of the reason that HTNs have been widely adopted. 


2.2 Behavior Trees 

Behavior trees (BT) are a means of achieving artificially intelligent behaviors pop¬ 
ularized in the video game industry. Their expressiveness, extensibility, and ease 
of use have led them to gain popularity over other, less extensible methods such as 
FSMs [20, p. 159]. Although the exact origin of behavior trees is unclear, the first 
mainstream game in which behavior trees were used was Halo 2, which was released 
in 2004 [21]. 

2.2.1 Behavior Tree Basics 

BTs are composed of nodes and the connective branches. Nodes represents either a 
task or structural syntax of the tree, and are classified as one of three general types: 
composites, decorators, and actions [22, p. 262], Composites and decorators define 
the logic and syntax of the tree, while the actions run code which typically aims to 
change the game state. BTs are read from top to bottom and left to right, and use 
a depth-first logic. Every node in a BT either returns success or failure to its parent 
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node, even if it takes some time to execute. Traversal of a branch ceases when a 
node returns failure. BTs can be as expressive as the language in which they are 
implemented, and are inherently more modular than simple scripting alone [23]. 

The nodes discussed in this section are as general as possible, but apply most directly 
to the BT implementation in Opsive’s Behavior Designer, version 1.5.5, in Unity 
version 5.1 [24], 


Composites 

Composite nodes determine the manner in which their child nodes will be executed. 
There are three primary types of composite nodes: sequences, selectors, and parallels, 
as seen in Figure 2.4. 



Figure 2.4: Three major types of composite nodes. Source: [24], 


• Sequence nodes. Sequence nodes execute their child nodes in left to right 
order, returning “success” back up the tree only when all child nodes are suc¬ 
cessfully completed. They will immediately return “failure” to their parent node 
if any children fail. In this way, they are similar to "and" logic, as all children 
must succeed for the parent sequence node to succeed. 

• Selector nodes. Selector nodes also execute their child nodes in sequence, left 
to right, but return success as soon as any child returns success. In this way, 
they are most similar to "or" logic, as only one child needs to return success to 
the parent selector node in order for the selector node to itself return success. 

• Parallel nodes. Parallel nodes are the exception to the “rule” that BTs are 
read from left to right, because parallel nodes execute their child nodes at the 
same time. They are similar to the sequence nodes in that they only return 
“success” when all child nodes succeed; otherwise they return “failure.” 
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Decorators 


Decorators are essentially composite nodes with only one child [22, p. 263]. Decora¬ 
tors do things like repeat a subordinate branch until a condition is met, or provide 
a way to interrupt the branch below it from another part of the tree. Examples of 
decorator nodes are shown in Figure 2.5. 



Figure 2.5: Examples of decorator nodes. Source: [24], 


Conditionals 

Conditionals restrict the flow of execution within a behavior tree in exactly the same 
way that if-else or switch-case statements do in traditional scripting. Their contents 
must eventually resolve to a Boolean. To be useful, they typically have a sequence 
as a parent and one or many action nodes as successors. This allows conditionals to 
act as gatekeepers to action-oriented nodes. Some basic examples of conditionals are 
shown in Figure 2.6. 



Figure 2.6: Examples of conditional nodes. Source: [24], 


Actions 

Action nodes are the meat of BTs, as they are the only nodes which are meant to 
change the world state. The BT itself may have many nodes of complex logic in 
conditionals, decorators, and composites, but without action nodes, nothing would 
ever happen. The possibilities for action nodes are limited only by the programming 
language and the application domain, but typically describe tasks such as attack, 
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move, crouch, or send a message. They may also include tasks that change variables, 
store and represent knowledge, interrupt another task, or make calculations for other 
purposes. It is important for any implementation of BTs to include extension options 
for authors to build custom actions for their application. 
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Figure 2.7: Examples of action nodes. Source: [24], 


There is no limit on the code included within an action node. They may range from 
getting or setting a local variable to executing a complicated detection algorithm. 
This can be a potential pitfall, as nodes which represent complex tasks may be better 
broken down into several more simple nodes and then combined into a tree. They can 
then be inserted into the original tree not as a task, but as a behavior tree reference. 


Behavior Tree References 

BTs are inherently modular, meaning that they can be pieced together to make more 
complex BTs. Often this is accomplished with behavior tree references, where an 
entire BT is treated as a child node within another BT. This allows for efficient 
code re-use, because when the child BT referenced in the parent BT is changed, the 
parent BT does not have to be edited. In this way, behavior tree references leverage 
the same type of convenience and efficiency that is the cornerstone of object-oriented 
programming (OOP). 

2.2.2 GoToStore as a Behavior Tree 

This section details one possible implementation of the GoToStore HTN from sec¬ 
tion 2.1.5 as a behavior tree within Unity 5.1, using Behavior Designer version 1.5.5. 
Chapter 6 details the making of BTs in Unity; this tree serves only as a syntactic ex¬ 
ample. The objectives are to demonstrate through simple example the syntax of BTs 
and to provide an example suggesting that HTN-like constructs can be represented 
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as BTs. We will be using the basic components detailed in the previous section to 
walk through the BT shown in Figure 2.8, assuming the following starting state: the 
agent is at home and has the keys to his car. The initial execution of the tree can be 
seen by following the trace described by the numbered nodes in the image, which is 
detailed in the following section. 



Figure 2.8: GoToStore Implemented as a BT in Opsive's Behavior Designer 


1. Execution of the BT begins at the entry node, and proceeds to the repeater. 
The repeater ensures that this BT will not stop executing unless explicitly told 
to. Without the repeater node, the tree would stop after passing through all 
branches exactly once. 

2. The next node is a "selector" node, which starts with its leftmost child. At the 
abstract level, this node dictates that the agent will only check for keys to drive 
to the store if it is at home. Otherwise, it will simply walk to the store. 

3. This sequence node will check its children from left to right, which in this case 
is checking if the agent is at the store. 
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4. "Within distance" is a conditional node, and fails here because the agent is at 
home. It returns failure back to the parent sequence node labeled "3," which 
returns failure to the selector node labeled "2." 

5. Execution passes to the next child of "2," which is another selector node. 

6. This node represents the notion of checking to see if the agent is at home. It 
passes execution to its first child. 

7. "Within distance" is a conditional node, which succeeds because the agent is at 
home. Because its parent is a sequence node, execution passes to its successor, 
also a child of node "6." 

8. This selector node passes execution to its first child. This branch must be 
explored before the "Walk To Store" node is visited. This structure implicitly 
biases driving to the store over walking to the store. 

9. Another sequence node, which passes execution to its children. 

10. The conditional "Has This Child" is checking to see if the key is attached to 
the agent. It is, so this node returns success to its parent. 

11. The agent then executes a movement task which directs it to get into the car. 
This task has been completed at the point in time represented here. 

12. The BT is currently executing the "Drive to Store" task. Upon completion, 
it will return success back up the tree through nodes 9, 8, 6, 5, and 2, to the 
repeater node. 

13. The repeater node then restarts evaluation of the tree, which finds that the 
agent is at the store. This time around, node 4 will return success, and node 
13 will be visited, which stops the tree. Execution is complete. 


In the same way that HTNs can be pieced together to create complex behaviors, this 
BT could also be built into more complex behaviors. 

The following graphic, Figure 2.9, overlays the HTN nodes from Figure 2.2 on to the 
BT shown in Figure 2.8. In several cases, one node in the HTN equates to several 
nodes in the BT. For example, the sequence node labeled “2” and the conditional 
node labeled “3” combine to create the equivalent of the “At Store” constraint node 
from the equivalent HTN. Chapter 6 details the translation of BTs to HTNs, using 
both generic equivalencies and more complex examples. 
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Figure 2.9: GoToStore HTN Overlayed onto BT 


2.2.3 Using BTs to Emulate HTNs 

The expressiveness of BTs means that their implementation can be as unique as 
needed. This also means that there are likely many ways to achieve a desired be¬ 
havior. For example, one designer may design a tree whose first node is a parallel 
composite node. This means that each branch below the composite node will execute 
simultaneously. Another designer may build those two branches into two separate 
trees, and run them simultaneously. This effectively achieves the same behavior, but 
may appear significantly different. Yet it is the expressiveness of BTs that led to 
their selection for use in this thesis. After all, we are attempting to emulate a specific 
implementation of HTNs, and BTs provide the most versatile tool with which to do 
so. HTNs are a better fit for the precision needed in COMBATXXI than their be- 
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havior tree counterparts. However, while some major game development companies 
have used HTNs, there are not many implementations of HTNs available for use in 
the developers’ tool market. 
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CHAPTER 3: 

Managing Dynamic Behaviors in COMBATXXI 


COMBATXXI is an agent-based simulation of combat, falling into the second high¬ 
est of Dr. Ilachinski’s eight tiers of complexity of combat models [3, p. 237]. 
COMBATXXI is one of the most detailed agent-based simulation of combat to date, 
and includes models of many of the complex processes which occur in combat, such as 
target acquisition, ballistics, variations in terrain mobility, tactical communications, 
and much more. The combat model itself has over 750,000 lines of code, and the 
primary development environment, Scenario Integration Tool Suite (SITS), is equally 
large. However, with this high level of fidelity comes complexity in building scenarios, 
an increase in the time required to run them, and more complex output data through 
which users must sort. Much of the complexity in COMBATXXI is not needed to 
test the basic behavioral logic of entities. The primary benefit of COMBATXXI-high 
fidelity-may inadvertently be the primary barrier to developing and managing agent 
behaviors. 


3.1 Testing Behaviors in COMBATXXI 

Testing agent behaviors in COMBATXXI is no trivial matter. Most analysis in 
COMBATXXI is conducted with the output data, which comes in two main forms. 
The largest form of output data is the "loggers". Although loggers can be difficult 
to sort through, they are the most reliable form of data output from COMBATXXI. 
Users can also utilize visual representations of troop movements and some events 
during scenario execution, but this is limited. 


3.1.1 COMBATXXI Loggers 

Loggers are typically .csv hies that save certain specific types of data, and are only 
saved when specified in the scenario manager. There are currently 33 loggers that 
users can choose from, some of which have many more subordinate loggers with more 
specific information. The interface used for selecting loggers is shown in 3.1. 
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Figure 3.1: Loggers which save simulation run data are selected by the user. 


Automated data processing tools are used by OAD and Training and Doctrine Com¬ 
mand Analysis Center, White Sands Missile Range (TRAC-WSMR), the primary 
consumers of COMBATXXI. These tools may be built using common desktop soft¬ 
ware such as Microsoft Excel, or more flexible software such as Java. However, most 
of these tools are meant for post-processing of scenario output data, and do not focus 
on behavior debugging. 


3.1.2 Visual Output 

COMBATXXI output can be visualized with two different tools: a "viewer tool and 
a "3D viewer" tool. The viewer can be used while a simulation is running, or it can 
be used to view simulation output after scenario execution. The use of these tools 
within the work flow for a COMBATXXI scenario designer may proceed as detailed 
in the following sections. 


3D viewer tool 

The 3D viewer tool is primarily meant to visualize spatial orientation of terrain, 
obstacles, and buildings, and is meant to be used before running the simulation. A 
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scenario designer may use pre-existing terrain data, or they may build it on their 
own using other tools within the COMBATXXI architecture. The designer may use 
the 3D viewer at multiple points during the scenario development to ensure initial 
placement of entities is rational and as intended. There is no ability to use the 3D 
viewer during the simulation run or to view Distributed Interactive Simulation (DIS) 
data afterwards. An example of what a designer may see in the 3D viewer is shown 
in Figure 3.2. The 3D viewer was built to help terrain designers, who don’t have the 
requirement to move around the scene as much as a behavior designer would. As a 
result of its initially intended purpose, it is not often used by behavior designers. 



Figure 3.2: 3D Viewer tool in COMBATXXI. Source: [5]. 


Using the Viewer Tool 

Before using the viewer, it is important to understand that it is portraying recorded 
DIS Protocol Data Unit (PDU)s as specified in the IEEE Standard 1278.1 [25]. This 
standard is meant to promote interoperability across simulation platforms, and is a 
natural fit for COMBATXXI. However, this standard can be limiting, for example, 
when soldiers have multiple weapons. The DIS standard only allows for the soldier’s 
primary weapon system to be represented in a DIS PDUs, as shown in the soldier’s 
prohle in Figure 3.3. Additionally, the scaling of entities within the viewer is arbitrary 
and user-defined, so entity geometry is not accurately portrayed. There is no “rewind” 
functionality in the viewer-play and pause are the only options. 

In spite of its apparent limitations, the viewer can be a useful tool for debugging a 
scenario. It does show some events, such as direct and indirect fire engagements, as 
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Figure 3.3: DIS enumerations are manually specified in the entity profile 
tab in SITS. The "SubCategory" field only holds one value for one weapon 
system. 

well as fatalities. If COMBATXXI users choose to view the output from a single 
simulation run at run-time, it is typically done with the model output window open 
adjacent to the viewer. This allows users to see output and error statements, as 
shown in Figure 3.4. However, this output comes across the screen as fast as the 
simulation can be processed. This may be extremely fast in comparison to real-time 
if the simulation is relatively simple, or it may move slower than real-time if it is 
complex. 



Figure 3.4: Viewing the model output panel adjacent to the viewer during a 
simulation run. 

In practice, COMBATXXI users typically need to run many iterations of a simula¬ 
tion, and can do so from the "run manager" window without seeing each iteration’s 
DIS output visualized. They then may analyze the output data using their preferred 
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method, and open the viewer retroactively only when something needs further inves¬ 
tigation. Using the viewer retroactively produces the same result as viewing the DIS 
output at run time. 


3.2 The Need for Robust Testing of Behaviors 

Early testing is a crucial habit to prevent small bugs from causing large problems later 
in development [9, p. 238]. It is best to test often when developing code so that when 
bugs do occur, developers have a better idea of what may have caused them. This 
also prevents the developer from having to untangle the effects of having to change 
some earlier mistake on the rest of their code. This concept is equally applicable in 
COMBATXXI due to its complex structure. However, for all the reasons mentioned 
thus far, testing early and often in COMBATXXI is not easy. 

3.2.1 Test Scenarios 

Building specific scenarios for a variety of test cases is time-consuming manual work. 
For example, the basic scenario authoring tutorial is recommended to take 12 hours, 
and the terrain is already set up [26]. Building a set of scenarios with varying terrain 
is an advanced skill in COMBATXXI. For example, simply changing the orientation, 
size, or number of buildings in a given area requires new terrain to be built. This 
difficulty is prohibitive for developers who want to test their behaviors in a wide 
variety of terrain orientations. 

The discussion in Chapter 1 regarding the nature of complexity theory focused on 
the inherent benefit of leveraging the complexity that emerges from simple behaviors, 
or the simple natures which emerge from complex behaviors [3, p. 5]. COMBATXXI 
aims to leverage the latter type of emergence, but the resulting macro-level behaviors 
are not always convergent. Sometimes, behaviors in COMBATXXI lead to divergent 
or at least unpredictable behaviors. This may be due to inconsistencies in the behav¬ 
ioral logic or the unpredictability of the external input to the behavior. Often, such 
inconsistencies arise later on the in the simulation, when the possibility for a wide 
range of world states prior to the instantiation of the behavior increases. 

These are the most difficult behavior-related problems to solve in COMBATXXI, 
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because a simulation may take a long time to arrive at the point where the behavior 
is instantiated. The causal stimulus may not arise in each iteration of the simulation, 
further complicating the debugging process. Gathering data and recognizing a pattern 
that may be indicative of the underlying cause of the problem may take many more 
simulation iterations than the behavior designer has time to perform. One possible 
solution to this problem could be to build smaller scenarios which are similar to the 
scenario at hand, but this can be similarly time-consuming. 

3.2.2 Modular Testing 

It has been suggested that testing code in a manner similar to testing integrated 
circuits is a beneficial practice for software projects [9, p. 189]. That is, each atomic 
module of code should be tested on its own first, and then tested as a larger whole. 
This is a logical, and even intuitive way to test BTs and HTNs, particularly because 
a trademark of well-built behaviors is modularity. 

Due to the potentially explosive complexity of agent behaviors in COMBATXXI, 
all possible emergent agent behaviors cannot be reasonably predicted through obser¬ 
vation of the code alone. As with any software, early and ruthless testing are key 
to a successful outcome [9, p. 237]. Therefore, a thorough verification and valida¬ 
tion (V&V) routine should include a wide array of test cases that diversely challenge 
and completely encapsulate the behavior requirements. Ruthless testing of any soft¬ 
ware helps to confirm two things: whether the code satisfies the requirements, and 
whether the requirements match the customer’s intent. Working with subject matter 
experts to develop realistic test-cases may help to identify problems with individual 
behaviors before they are brought in to already complex scenarios. 


3.3 Options for Improved Testing in COMBATXXI 

There are three options going forward for behavior development in COMBATXXI. 
The status quo can always be maintained, and should be considered. If improvement 
efforts incur a significant manpower investment, the status quo may be the only 
tenable option. The other two options include development work, either by way 
of extending COMBATXXI’s current development environment, or by building a 
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surrogate test environment for behavior developers. 


3.3.1 Extending COMBATXXI 

Extending COMBATXXI should not be ruled out. The primary benefit of investing 
development efforts to extend the COMBATXXI development environment is that 
those improvements would reside in the same code base as COMBATXXI. This would 
result in more direct translation of development efforts, as opposed to a surrogate 
environment from which behaviors must be translated. 

There are three ways in which COMBATXXI could be extended. First, the viewer 
could be made to incorporate a modern 3D viewer-combining the top-down viewer 
and the current 3D viewer into one. The ideal 3D viewer would be comparable to 
modern game development environments’ viewers. This could include more direct 
control of the playback speed, as well as a rewind option. A second option could 
be to build higher-level abstractions within COMBATXXI for current models which 
currently require burdensome detail. Lastly, reusable prototypes could be built which 
allow designers to quickly deploy highly complex networks of agents. An example for 
the latter two could be the use of indirect fires (IDF), which currently requires the 
existence of a unit to fire the IDF, a fire control center, and a forward observer. If a 
scenario designer only needs the effects of IDF, it would be useful to have an easily 
deployable IDF zone which can be easily initiated from a behavior. 

3.3.2 Testing in a Surrogate Environment 

The final option, and the focus of this thesis, is the development and testing of behav¬ 
iors for COMBATXXI in a surrogate environment. The following chapter examines 
best practices from the commercial video game industry and how those methods may 
benefit the development of dynamic behaviors. 
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CHAPTER 4: 

Best Practices from Commercial Game Development 


In 2015, the commercial video game industry achieved a record $61 billion in world¬ 
wide sales, split between mobile, PC, and console games [27]. Although traditional 
console games are losing market share as a percentage, individual console games have 
brought in incredible amounts of revenue in recent years. For example, sales of Grand 
Theft Auto V, which was released in September 2013, totaled more than $1 billion 
in the three days following its launch, and high-grossing releases are no longer the 
exception. These major titles are incredibly complex, and require large amounts of 
investment up front to develop. Grand Theft Auto V, for example, reportedly cost 
over $265 million to produce [28]. More recently, it is estimated that Final Fantasy 
XV will have to sell 10 million copies just to break even after its release in late 2016. 
Considering that only one Final Fantasy release has sold that many copies thus far, 
this increased acceptance of high development costs is representative of the fact that 
the market for these types of games is still growing [29]. With this increased spending 
in commercial game development, it is reasonable to believe that there may be best 
practices for development which could be applied to particular challenges within the 
DoD M&S community. 

Military simulations face many of the same development challenges as commercial 
video games, such as optimization, debugging, and code re-use. High-resolution mili¬ 
tary analytic simulations include detailed physics and system models of combat equip¬ 
ment such as tanks and artillery, however, cognitive or behavioral representation is 
more of a challenge. The concept of maneuver warfare has represented a challenge 
for the DoD M&S community since the advent of agent-based simulations and the 
corresponding divergence from attrition models [3, p. 25]. While military doctrine 
can serve as a starting point for maneuver-related behaviors, different subject matter 
expert (SME)s may have varying interpretations of that doctrine. This complicates 
the development and validation of maneuver-based behaviors, and military agent- 
based behaviors in general. Further complicating the task of behavior developers, the 
majority of military operations of the past decade have involved irregular, or asym- 
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metric warfare. Modeling these types of conflict is even more complex than traditional 
force-on-force combat [30]. 

However, at the core of these complex military simulations is the agent, be it a 
soldier, a tank, an airplane, or an insurgent. Nearly all commercial games require 
some form of artificially intelligent agents to act as enemies, companions, or other 
non-player characters, and all of those agents require behaviors. It is reasonable to 
posit that some best practices from the commercial game industry may be useful for 
the development of agent-based behaviors for military simulations. Adopting the most 
applicable methods may improve "modelers’ ability to explore possibility spaces and 
potential outcomes" [31]. This chapter provides an overview of the techniques used 
in the commercial game development industry and the software engineering industry 
at large as they may apply to the development of dynamic behaviors for military 
simulations. 


4.1 Applicability of Game Development Techniques 

The increasingly large budgets of commercial video games makes it ever more likely 
that cutting-edge development techniques will emerge from the commercial domain. 
However, this does not mean that all techniques developed for real-time video games 
will transfer well to high-resolution, multi-agent based simulations of combat. In 
order to explore the applicability of commercial game development best practices to 
the development of behaviors for multi-agent based simulations of combat, we must 
first identify the traits which these two development domains have in common. 

4.1.1 Analytic Simulations’ Similarities with Video Games 

The similarities between game development and simulation development far exceeds 
their differences. The fact that they are both rooted in highly complex code bases 
means that much of what applies to general software engineering applies to both game 
and simulation development. Both domains at least partially rely on OOP, behavior 
development and re-use, and managing complex models. 

Object Oriented Programming. OOP is at the core of both game and simulation 

development. For example, the highly regarded Game Programming Patterns , 
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by Robert Nystrom, is essentially a modern, game development-focused update 
to the classic Design Patterns: Elements of Reusable Object-Oriented Software 
[8, p. 4], Design Patterns, which was first published in 1994, is viewed as a 
classic for OOP development. If the fundamentals of OOP are still useful for the 
development of commercial games, it is likely also useful for the development 
of military simulations. 

Behavior Development and Re-use. Behavior development can be a tedious pro¬ 
cess, and as with any software project, re-use can help to reduce the need for 
work duplication. This concept may fit in with the OOP paradigm, where two 
classes of agents which inherit from the same parent class behave in very similar 
ways. This concept applies to military simulations as well. For example, a tank 
and an armored personnel carrier are both types of vehicles, and should have 
some similarities in the way they behave. Modelers should not need to design 
completely new behaviors for actions, such as movement, which are basically 
similar between those two classes. Another way in which code re-use benefits 
military analytic simulations is the applicability of a behavior to multiple ech¬ 
elons of units. For example, an ideal maneuver behavior which orders a squad 
to move in a wedge could also be used to order a platoon to move in a wedge. 

Managing Complexity. With approximately 1.5 million lines of code in the 
COMBATXXI development environment, and hundreds of millions of dollars 
going in to commercial game development, it is not a stretch to say that both 
domains have to be adept at managing complexity. Challenges that arise in 
highly complex software projects include version control, geographically dis¬ 
persed development teams, interoperability, and architecting for forward com¬ 
patibility. While both OOP and behavior re-use help manage this complexity, 
there are myriad other solutions that can help to streamline the development 
process. 


Both the commercial gaming industry and the military simulation community demand 
reliable implementations of models. Bugs in games can prevent them from being 
distributed on a given platform, and errors in simulations can cause crashes that 
result in the loss of hours of processing and debugging time. Both industries leverage 
the OOP paradigm for code reuse and extension. However, that is mostly where 
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the similarities stop, at least within the specific domain of military simulations for 
analysis, where 3D visualization is not a priority. 


4.1.2 Analytic Simulations’ Differences with Video Games 

The differences between commercial games and analytic simulations are important to 
note so that reasonable assessments can be made regarding which methods may not 
transfer well from the commercial game industry. 

Number of agents. Typical, large-scale military simulations of combat include mil¬ 
itary units up to the Brigade level. A Marine Corps MAGTF, such as a 
ship-based Marine Expeditionary Unit (MEU), consists of approximately 2,300 
Marines. The number of agents in a military analytic simulation will be greater 
than a typical commercial game. Therefore, it is beneficial to be aware of the 
scaleability of a given technique. The OOP nature of analytic simulations ac¬ 
counts for this, and the fact that analytic simulations can be run slower than 
real-time means that scaling will only increase the run time. 

Time-step vs. Discrete Event. Modern video games revolve around the game 
loop pattern, which updates the state of the game as much as possible be¬ 
tween each frame rendered [8, p. 128]. Typically, this means that the state of 
the world is updated 30 or 60 times per second under ideal conditions. The sim¬ 
plest variety of this pattern is depicted in Figure 4.1. Contrastingly, constructive 
military simulations such as COMBATXXI leverage discrete event simulation 
in order to maximize event timing precision, minimize computational resources, 
and decouple the simulation state update process from other processes, such as 
graphics rendering process. 



Figure 4.1: Simple Game Loop Pattern. Source: [8] 
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Graphics focus. Commercial video games are able to use more realistic graphics 
with each new iteration of hardware. Graphical representation of the game 
world is often the primary focus of commercial games. This means that a good 
deal of the computational resources must go to the "render" portion of the Game 
Loop depicted in Figure 4.1. COMBATXXI does not necessarily produce any 
graphical output, unless specified by the user. This is due to the intended use of 
COMBATXXI scenarios, which are typically run dozens of times in order to pro¬ 
duce analytically tractable output. In an already complex simulation, coupling 
the simulation state update process to a rendering process would needlessly, and 
likely excessively slow down the output production process. However, the user 
may request visual representation before runtime, and DIS PDUs are logged 
and played back on a top-down map in order to spatially depict the events in 
the simulation-but only after they have occurred. Section 7.2.2 provides sug¬ 
gestions for future work which may leverage commercial game engines’ graphics 
focus for the betterment of COMBATXXI model validation and analysis. 

User input. Video games rely on user input in order to drive the gameplay. With¬ 
out user input, there would be no game. Most constructive simulations are 
closed-loop, meaning that no user input is provided during run-time, or only 
have limited high level command input. This puts more of an impetus on the 
fidelity and realism of the agent behaviors in multi-agent based constructive 
simulations, but does not create any additional barriers for the use of commer¬ 
cial game development best practices. 

Deployment Environment. Video game development teams are often tasked with 
deploying a game on to multiple different platforms. This requires foresight 
when architecting the game development environment that may not be nec¬ 
essary for the DoD’s analytic simulations. In the case of COMBATXXI, the 
developers and the end users are often in direct communication, and they are 
able to ensure that new hardware purchases or potential changes to the archi¬ 
tecture will not adversely affect the ability to use the model. 


These differences do not mean that game development practices related to them will 
not be useful for the development of behaviors in analytic simulations. Instead, they 
serve as context to help focus the search for the development mechanisms which are 
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most likely to be applicable. 


4.2 Identifying Applicable Best Practices 

As with any software design, the commercial game development industry is subject to 
the sirens’ call of buzz words, alleged short cuts to game design success, and fleeting 
design fads. Additionally, there are likely to be some highly successful techniques 
which simply do not help development of agent-based behaviors due to the reasons 
discussed in the previous section. Therefore, it is the imperative of simulation design¬ 
ers to attempt to qualify attributes of development techniques that will most readily 
apply to the development of agent-base behaviors. Here we examine those qualities of 
a behavior development methodology which may contribute to the betterment of the 
behavior development process, thereby increasing quality and decreasing development 
time. 


4.2.1 Higher Quality Behaviors 

Defining exactly what makes a behavior have higher quality is neither an exact nor 
an agreed upon science. However, a set of attributes has been suggested in previous 
thesis work at NPS to identify high quality behaviors [10, p. 8]. Here we examine 
those attributes with the intent of examining whether or not a given development 
methodology will lend itself to producing behaviors with these qualities. 

Creation: Is the methodology sufficiently expressive to allow for the creation of 
complex military behaviors? 

Efficiency: Does the methodology help to streamline the scenario development pro¬ 
cess, reducing the overall production time, and improve the ability to commu¬ 
nicate the intent behind the behavior to other potential users? 

Modularity: Does the methodology lead to behaviors which can be composed into 
progressively more complex behaviors? 

Reusability: Does the methodology create behaviors which can be reused in other 
scenarios? 

Scalability: Can behaviors built with this method be used at different echelons of 
military units within the simulation? 
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Although poorly written behaviors can be created in the best of development pro¬ 
cesses, a limiting development process can prevent the above criteria from being 
achieved even by experienced authors, ft is with this in mind that we seek out the 
most enabling development methods and tools for our dynamic behavior development. 

4.2.2 Faster Development Cycle 

Aside from the overall quality of a behavior, the other most significant question to be 
asked of a development process is whether or not it helps to reduce the development 
time. Developing, distributing, and providing requisite instruction for a behavior 
prototyping environment requires a potentially large amount of effort up front, so it 
needs to provide significant development time reductions with increased use. If simple 
behaviors are needed for simple scenarios, the efforts required to build a prototyping 
environment may outpace their benefit. 

While development time is more practical to measure than behavior quality, conduct¬ 
ing a full user study is beyond the scope of this thesis. Although assessment of a faster 
development process is somewhat arbitrary, the following sections in this chapter aim 
to qualify characteristics of a behavior development environment or methodology 
which may help reduce the time required to develop new dynamic behaviors. 


4.3 Benefits of 3D Development Environments 

Cognitive modeling of agent behaviors is a difficult task to undertake. Similar to plan¬ 
ning in real-life, unintended stimuli have a tendency to throw our plans, or in this case 
our agents’ behaviors, off track. Due to the highly complex nature of COMBATXX1, 
it is not likely that our behaviors will work the first time. We will need to debug them, 
and to do so we must first grasp exactly what is breaking them. COMBATXX1 users 
can leverage the loggers, the runtime output, or the visual output to do this, but 
game designers are able to leverage the visual nature of their games to design and 
debug. 

Wallach, O’Connell, and Neisser’s article about the so-called "kinetic depth effect" 
in the Journal of Experimental Psychology in 1953 posited that three-dimensional 
views of complex scenes enabled users to better understand what was happening 
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within those scenes [11]. Recent research into understand how people retain three- 
dimensional information has corroborated this hypothesis, arguing that this ability 
"enables the user to obtain a different view suitable to the task, reveals structures 
that may otherwise be occluded, and allows better perception of rigid 3D structure, 
through structure-from-motion, also called the kinetic depth effect" [12]. 

A similar logic applies to the development of artificially intelligent behaviors which 
play out in a three dimensional environment. As an example of this concept in 
the commercial game industry, the AI development team for Sunset Overdrive at 
Insomniac Games was able to visually debug their game’s enemy behaviors within 
the levels in which they would be employed. This included visual representation of 
path planning and other parameters needed to fine tune the behaviors [32] of the 
game’s antagonists. Had they needed to rely only on textual output, this certainly 
would have been a more difficult process. 


4.4 Prototyping Models and Behaviors 

In order for a prototyping environment to be useful, it must eventually save the 
developer time. In order to do so, it should be as fluid and dynamic as possible, 
enabling the developer to implement many designs in a short amount of time in order 
to fold the most suitable design for the end use simulation. An ideal prototyping 
environment enables designers to safely explore combinations of models and behaviors 
without breaking prior existing models and behaviors. 

Game development tools enable developers to easily reuse models that have already 
been built and quickly develop new models based on previously existing models. In 
order to accomplish this, three common concepts may be used: the object-oriented 
programming paradigm, the blueprint pattern, and the sand box pattern. 

4.4.1 Object Oriented Programming for Prototyping 

Object Oriented Programming (OOP) is a cornerstone of many software projects, 
including COMBATXXI, and it is worth exploring the benefits of OOP in relation to 
prototyping and transferring behaviors. OOP is more likely to be useful in a proto¬ 
typing environment if the end use environment, in our case COMBATXXI, leverages 
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OOP. Mimicking the class structure of the end use environment will assist with 
transferability when a behavior is finalized. 

Deviation from mimicking of the end use class structure should only be done when 
absolutely necessary. At best, it will make the transfer process more difficult and error 
prone. At worst, it may lead to behavioral constructs in the prototyping environment 
which don’t transfer to the intended simulation environment. 

For example, lets imagine that a machine gunner entity in inherits from the basic 
soldier entity in our end simulation. The majority of the functionality of the machine 
gunner is based on that of the basic soldier entity, and they share many of the same 
variables. If our prototyping environment develops the basic soldier and the machine 
gunner such that they have different public and private methods or variables, this 
could create unnecessary confusion or even lead to an infeasible transfer. 

4.4.2 Reusable and Dynamically Adjustable Components 

Game engines typically allow designers to build or import a model once and then use 
it many times within the game. These objects are called "blueprints" in the Unreal 
Development Kit, and "prefabs" in Unity3D [33]. 6 For clarity, the concept will be 
referred to as prefabs from this point forward. 

Prefabs are like blueprints to a model. The blueprint exist as a concept, but until 
it is brought into the simulation, it has no functional purpose. Prefabs are reusable, 
different instances of the prefabs may have different parameters at different points in 
time. For example, if a soldier is implemented as a prefab, the prefab may start out 
with 100 level health. Obviously each instance of the same prefab may have different 
levels of health. In this way, each instance of a prefab comes from the same source 
but exists as its own object. 

Prefabs may often be similar to one another, and therefore it can be easy to build a 
new prefab based on a prior existing one. This is similar to classes in OOP with one 
key difference. A prefab which is based off of a prior prefab will break all ties with 
that previous prefab. 

6 More on Unity prefabs can be found here: http://docs.unity3d.com/Manual/Prefabs.html 
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This is a somewhat similar to profiles in COMBATXXI, with one key difference. In 
SITS, the representation of an entity must be placed on the force structure board 
prior to a profile being applied to it, while a prefab typically encapsulates the entity 
and the profile. They are similar in that they are both easy to make new deviations 
from prior existing profiles, and the new profiles can be independent of the original 
profile from which they were built. 

This concept helps behavior developers to quickly alter, test, and revise behaviors 
that are attached to agents in the prototyping environment in a manner that does 
not affect previously built agents models or behaviors. If the new design is preferable 
to the old design, it can be made to replace it outright, or replace it in future designs. 

4.4.3 The Sandbox Pattern 

The "Sandbox" pattern is another way of providing a wide range of stable yet dy¬ 
namic options for game designers. The example provided by Robert Nystrom in 
Game Programming Patterns of interchangeable "spells" which a hypothetical game 
uses to allow wizards to compete is similar to our desire to rapidly experiment with 
entity-level behaviors in COMBATXXI |8]. Game designers and players alike can ex¬ 
periment with different combinations of spells without breaking the game or inducing 
unforeseen bugs to the code. 

This is similar to the concept of modularity in behavior development for simulations. 
Any given behavior, whether represented as a behavior tree, a hierarchical task net¬ 
work, or any other abstraction, will likely be composed of many sub-tasks. Ideally, 
those sub-tasks have the ability to stand alone or be combined in different combina¬ 
tions in a variety of situations without inducing bugs. This concept allows behavior 
designers to rapidly experiment with a variety of behavioral logic that meets their 
requirements. 

The sand box pattern can be used throughout a game’s design, including the area of 
behavior design. For example, game designers may want players or other designers to 
be able to build new characters. Perhaps the designers want to be able to easily swap 
out masks, clothing, or other visual components of their characters. The sand box 
method would build all these potential elements and allow the user to safely swap 
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them out without breaking anything. 


The sand box pattern may be used in commercial games in abstract low-level code in 
to higher-level tasks. These higher-level tasks can then be combined in various ways 
by behavior designers who won’t have to interface with source code [8]. If scoped 
properly, these designers, working at the abstract level, can explore many different 
types of behaviors without touching any code or breaking any other behaviors. 

4.4.4 The Good in Bad Code 

Although not a primary benefit of prototyping in a parallel development environment, 
the required transfer of code from one environment to the other has collateral benefit. 
The notion that a behavior developer should expect to throw away their first design 
is widely held, as summarized in Nystrom’s Game Programming Patterns , and this 
section’s namesake [8, p. 14]. Discarding any development work may seem wasteful, 
and can be resisted by developers and managers alike. However, faster iteration cycles 
beget a faster initial development, and this makes the scrapping of the initial design 
less painful. 

The design schema proposed in this thesis involves writing behaviors in a program¬ 
ming language and software application which are different than the intended appli¬ 
cation. This method may seem inherently flawed by incurring the risk of transposing 
behavioral logic, yet it has potential benefits. Specifically, it ensures that prototypes 
must remain prototypes, and that no hastily built behaviors can be used as supposed 
production quality behaviors in COMBATXXI scenarios. Therefore, supposedly "bad 
code" is less likely to find its way in to production-level analytic simulations which 
will be used to make valuable decisions. 


4.5 Test Driven Development 

Test Driven Development (TDD) is a popular development technique in which no 
code is written without first building tests against which that code will be validated. 
The general process can be summarized by the following six steps [34]: 

1. Add a test to check one or several components of the specification. 
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2. Run each test in the test set. 

3. Write additional source code in order to pass the tests from step 2. 

4. Run each test in the test set again. 

5. Refactor the current source code to pass the tests. 

6. If not all specifications are met, return to step 1. 

As with any mature software development process, TDD for a behavior should not 
begin without a clear specification of the expected fidelity and level of detail to be 
included in that behavior. Fortunately, military doctrinal publications provide a 
standardized starting point for these behaviors. The use of military doctrinal publi¬ 
cations, when available for a specific behavior, allows for the generation and approval 
of detailed requirements which are well-suited for testing. Additional techniques for 
building tests may include soliciting vignettes from SMEs who may be able to encap¬ 
sulate information that bolsters the difficulty of the tests and the resulting robustness 
of the behavior. 


4.5.1 TDD as Applied to Prototyping 

While the end-validation of a scenario should be rigorously adhered to in the end 
use simulation environment, the TDD construct can be relaxed in the prototyping 
environment. The prototype behavior is built based on an abstraction of the model in 
which it will be eventually used. The entire benefit of the parallel prototyping schema 
is that it should speed up the process. Perhaps tests are written for the prototype, 
as in step 1 of the TDD process, but the designer building the prototype isn’t able 
to build the components for a behavior which meet those tests in a timely manner. 
It may be necessary to further abstract the prototyped behavior to move the process 
along. 

For example, imagine a designer is building a mounted patrol behavior. First, the sol¬ 
diers need to mount in the vehicle. The designer may initially want to explicitly model 
the movement of the soldiers to the vehicle in the prototyping environment. How¬ 
ever, the designers fords some development environment-imposed limitations which 
make this difficult, but he can "warp" them to be magically inside the vehicle. If the 
designer already knows how this "warp" will map to soldiers loading into the vehicle 
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in the end use simulation, time spent troubleshooting this is time wasted. Because 
the prototype is going to be discarded at the end of the process, a strict approach to 
TDD within the prototyping environment is not necessary. 


4.5.2 TDD on the Decline 

Although TDD is still a widely utilized programming paradigm, it does have valid 
and vocal detractors and should not be held as irrefutable gospel [35]. TDD is better 
utilized as a set of guidelines as opposed to hard-and-fast rules. For example, the 
behavior developer may not need to proceed all the way through the refactoring 
of current code to realize that they need to add additional code before the current 
code should be expected to work. As in all programming paradigms, common sense 
prevails, but TDD does provide a well-structured, forward-thinking paradigm that 
has the potential to enable developers to build more robust behaviors in a shorter 
amount of time. 

Regardless of potentially waning acceptance and use of TDD, it holds particular po¬ 
tential for the development of agent-based behaviors. Codingame.com is a popular 
website which allows novice and expert programmers to hone their skills by building 
agent behaviors which accomplish certain tasks in a video game application. Each 
application has an agent which can be programmed using one of several major pro¬ 
gramming languages. The user can see the tests in advance, and must write code 
to meet the tests. Each test adds an element of complexity, and each iteration 
of code development must account for this new complexity. In this way, users of 
Codingame.com are executing TDD without actually writing the tests themselves. 
Combining this construct with military doctrinal specifications as a source for tests 
makes TDD a promising candidate for use in the development of military agent-based 
behaviors such as the HTNs used in COMBATXXI. A final note on Codingame.com 
is that the programmer is able to immediately test their code and see the results of 
any changes that have been made. This fast iteration cycle is likely meant to keep 
users engaged, but is also a best practice for behavior development, as discussed in 
section 4.2.2. 
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4.6 Selecting the Most Applicable Methods 

The commercial video game industry, along with the programming community at 
large, provide myriad approaches for building a prototyping environment. Designers 
should leverage those methods which minimize their development time and maximize 
their ability to explore the design space and then transfer the behaviors to the end use 
environment. While some game development methods are meant to solve problems 
in less relevant areas such as graphics rendering and multi-platform deployment, the 
methods discussed here are applicable to the prototyping of behaviors for military 
agent-based simulations of combat. 
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CHAPTER 5: 

Unity as a Behavior Prototyping Environment 


Unity3D, known more simply as Unity, began development in 2001 as a small project 
by Unity Technologies-a company of only three developers. By 2005 it was released 
at Apple’s Worldwide Developer’s Conference, and quickly gained popularity as it 
leveraged an increase in game developer demand for a game engine which simplified 
deployment to multiple platforms [36]. By 2014, Unity had achieved a 45% share of 
the global game engine market, with over 4.5 million developers actively using the 
platform [37]. There is a free version of Unity that provides nearly full functionality, 
or a commercial version which costs $75 per month, and allows users to leverage 
advanced application deployment resources. 7 There is an active online community of 
developers and support personnel, a robust marketplace for reusable code and models, 
and an existing base of experts within the Modeling, Virtual Environments, and 
Simulation (MOVES) Institute at NPS. Unity was selected as the most appropriate 
game engine for use in this thesis for exactly these reasons. 8 


5.1 Introduction to Unity 

Unity is simple enough that users with only basic scripting experience can learn to 
build deployment-ready games in a reasonable amount of time. This is accomplished 
through an intuitive user interface, a robust online community and plug-in market, 
and an abundance of free tutorials built by the Unity team. 9 Certain concepts need 
to be understood to fully grasp the potential of Unity as a behavior prototyping 
environment. Many of these concepts will be familiar to game developers who have 
developed with other game engines, but some concepts are unique to Unity. These 
descriptions only cover the level of detail required to grasp the utility of Unity for a 
behavior prototyping environment. 

'Additional details on Unity3D licenses can be found here: http://unity3d.com/get-unity 

8 Unity 5 .3.1 was used for this thesis. 

9 Tutorials for all skill levels can be found at: http://unity3d.com/learn 
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5.1.1 Basic Unity Concepts 

The following sections cover only the key concepts within Unity which will help to 
understand its suitability for use as a behavior prototyping environment. For more 
in-depth explanations of the various aspect of Unity, please visit Unity3d.com. 


GameObjects 

GameObjects are the primary container within Unity that hold most other elements. 
Every GameObject must have a location, scale, and orientation, which are contained 
within the GameObject’s transform, but otherwise a GameObject may be empty. The 
position, rotation, and scale of a GameObject are contained within that GameObject’s 
transform, and transform hierarchies define the parent-child relationships within a 
scene. The parent-child relationship of GameObjects is what ties them together in 
the virtual environment. A child GameObject will change synchronously with its 
parent. For example, when a parent GameObject moves forward in the scene, the 
child will move along with it unless specifically programmed not to do so. 10 

A GameObject may be a plane which represents the ground or a mesh cube with a 
solid color material which represents a building. GameObjects may also be highly 
complex, containing not only the aforementioned elements which make the object vis¬ 
ible in the scene, but also scripts that contain detailed behavior-related information. 
The GameObjects used in the course of this work were kept as simple as possible, 
as the focus was placed on behavior development. Cinematic effects were only added 
where necessary for debugging. 


Game Loop Pattern 

At its core, Unity is a game development engine, and therefore revolves around the 
frame update cycle or game loop pattern, as discussed in section 4.1.2. It is sufficient 
for the purposes of this thesis to understand that Unity, or any other game engine’s 
reliance on the game loop means that simulations implemented in these game engines 
will update the world state once each frame and are therefore inherently time-stepped. 

10 For a detailed specification of the Unity transform, see the following: 
https: / / docs.unity3d.com / ScriptReference/Transform.html 
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This is not necessarily a limitation for our purposes, as no analysis is being conducted 
in Unity at this time. 


MonoBehaviour 

The primary enabler of GameObjects in Unity is the MonoBehaviour. Not every 
GameObject must have a script attached to it, but nearly every script that is attached 
to a GameObject is derived from the base class MonoBehaviour. 11 Scripting in Unity 
can be done in Javascript, Boo, or C#. Most people seem to use as their preferred 
language. 


Prefabs 

Prefabs are an asset type in Unity that acts as a template GameObject that designers 
can use for commonly reused GameObjects. Typically, a prefab is built by building 
it as an independent GameObject in the scene, and then saving it as a prefab. This 
preserves all components and characteristics of the original GameObject in a reusable 
form. A prefab is similar to a class, in that it defines what its spawned GameObject 
will be when it is instantiated. Once a prefab instance exists in the game, it exists 
as its own object, and may change its state without any ties to other instances of 
the same prefab. However, if the prefab is updated, it will affect each instance of 
the prefab in the scene, even if they have been changed. Prefab instances can break 
their connection to the parent prefab, resulting in complete autonomy. Any type of 
GameObject may be made into a prefab within a game. Players’ avatars, non-player 
characters, and inanimate objects may all be instantiated from prefabs. 

Prefabs also work as expected when the designer expects dimensional or parametric 
differences in the instances. For example, a designer may create a prefab of a generic, 
symmetrical box which represents a building, and then create a dozen instances of 
that prefab within the scene. He may then change the scale and rotation attributes 
of the building without "breaking" its connection to the prefab. Then, if the designer 
later decides to change the color of the building prefab, doing so will change the color 
of all instances of the prefab without erasing the custom dimensions that were already 

11 Details on all methods within the MonoBehaviour class can be found here: 
http://docs.unity 3d.com/ScriptReference/MonoBehaviour.html 
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applied. Only the element of the prefab which was changed resonates through to the 
prefabs’ instances. 


Assets 

The term "assets" in the context of Unity refers to basically any developed Unity ac¬ 
cepted hie type which can be transferred between users. This may include GameOb- 
jects, scripts, developer’s tools, entire scenes, or developer-specific hies such as "dll" 
hies which are required to enable a plug-in’s functionality. Unity provides a user- 
friendly container, known as a Unity package, for assets that are sold in the Unity 
Asset Store or exchanged between users. Assets are best kept in a central location 
and imported into a project only when needed. 


Project 

The Unity project is the container, represented as a folder, which holds all GameOb- 
jects, scripts, meshes, materials, prefabs, scenes, assets and other elements that are 
typically needed to produce a game. The project also defines the scope of the prefabs 
and scripts used in the project. This is important to note because any changes made 
within any part of a project to a specihc prefab, script, or other asset will affect all 
instances of that item within the project. This is not necessarily limiting, as a project 
can be duplicated, altered to experiment with some new or altered behavior, and then 
discarded if a developer doesn’t want to risk breaking other dependencies of a specihc 
item. Version control systems also provide protection from unintended consequences, 
and are widely used in the Unity developer community. 12 


Scenes 

Each game level in Unity is contained within a scene, which is a virtual environment 
in which the game is played. Scenes have their own world space and origin. Scenes 
may be linked together with a script that specifies their end conditions and order as 
in most games, or they may stand alone. An individual scene may contain everything 

12 For information on using Git, a source control standard for Unity development, visit this 
website: https://unity3d.com/learn/tutorials/topics/cloud-build/creating-your-first-source-control- 
repository 
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needed to prototype a set of behaviors, including terrain, obstacles, friendly and 
enemy units, and elements that enable the virtual environment, such as lighting. 

Scenes enable the use of the Sand Box Pattern as discussed in section 4.4.3, as they 
may be completely self-contained. This allows behavior developers to conduct rapid 
testing and try to break their behaviors before they are used in a production-level 
environment. 

5.1.2 The Unity Editor 

The basic Unity concepts are all brought together in the Unity developer’s user inter¬ 
face (UI), known as the "editor", which is a customizable, panel-based user interface. 
Unity is shipped with a set of standard windows in a default layout, but developers 
can build custom windows to make the editor more suitable for their specific work 
flow. The most common out-of-the-box windows in the editor are the scene, game, 
console, hierarchy, project, and inspector windows, as shown in Figure 5.1. These 
windows enable the basic functionality for development in Unity. 



Figure 5.1: A generic Unity editor configuration 

Starting with the upper left of the window in Figure 5.1, the scene window is one 
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of two visual representations of the scene. The scene window is not attached to 
any GameObject, and allows developers to view the scene from any position and 
orientation within the virtual environment. Through mouse and keyboard controls, 
users are able to virtually fly around the scene to examine the spatial relationships 
of various objects, or to zoom in on a specific part of the game. The scene window’s 
counterpart, the game window, is depicted in the bottom right. The game window is 
tied to a specific camera’s location and frame rendering parameters, and is typically 
used to provide game consumers with their character’s perspective, or to provide a 
static overview of the scene. 

The right half of the screen is occupied by the hierarchy, project, and inspector 
windows. The hierarchy window holds all GameObjects contained within a scene. 
The objects are organized by their transform relationships that create a hierarchy 
of GameObjects. Transform information is composed of a GameObject’s position, 
rotation, and scale. Changes to a GameObject’s transform are passed down to its 
children by default, but changes to a GameObject’s child transforms do not necessarily 
have any effect on the parent’s transform. 

The project window shows all hies contained within a project. This may include 
scenes, prefabs, 3D models, images, scripts, and other elements typically used to 
build a commercial game. It is similar to what is seen when opening the project 
folder from Windows Explorer. 

The inspector window depicts detailed information about whatever specific GameOb¬ 
ject is selected by the developer. This window is useful for run-time debugging, 
as developers can see the current values of public variables in scripts attached to 
GameObjects. Developers can also change those variables while running the game 
or simulation, allowing them to see how various behaviors change with parametric 
changes. The inspector window also serves to organize all elements of GameObject, 
such as scripts, animations, sounds, and navigation components. 

The console window is similar to consoles in other development environments. Unity 
allows scripts to print to the console using general messages or error messages, which 
are highlighted in red. 
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The Unity editor also allows users to play, pause, and progress the game frame-by- 
frame by using the three buttons centered at the top of the editor, as seen in Figure 
5.2. This is an excellent tool for allowing developers to see how their variables are 
changing frame-by-frame. This functionality also helps to debug many potential time- 
step simulation errors, where the precise time of detection or other events may be 
important [38]. 



Figure 5.2: The Unity pause, play, and frame-by-frame buttons 


5.2 Scripting with Unity 

Although Javascript and Boo can be used to write scripts in Unity, Ct^ was used 
for this thesis due to the author’s familiarity and an abundance of resident expertise 
within the MOVES Institute. This section discusses scripting specifically as it relates 
to the use of C# in Unity. 


5.2.1 Using MonoBehaviours 

Scripts written in typically derive from the MonoBehaviour class, and execute 
a series of methods inherited from that class with each update cycle. A full visual 
representation of the MonoBehaviour and the update cycle it represents can be seen 
in Appendix B. For the purposes of this thesis, the term "scripts" may be assumed 
to mean a script which inherits from the MonoBehaviour class. 

Although there are many methods which may be called during the MonoBehaviour 
lifecycle, the two most important are Start() and UpdateQ. The Start() method is 
called only once at the beginning of the script’s life, and is typically used to gather 
and set variables for use in the rest of the script. The Update() method is called once 
each frame, and is used to drive the script using the game loop pattern. 13 

13 Unity3D.com provides a full range of scripting tutorials in CA from beginner to advanced levels. 
They can be found here: https://unity3d.com/learn/tutorials/topics/scripting 
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Activating scripts 

Unity scripts will only take effect when they are attached to a GameObject which 
is active in a scene. GameObjects may be in a scene, yet deactivated. This is an 
important distinction for behavior developing. Developers may include many agents 
in a scene, and turn them "off" when debugging to simplify the scene and localize 
issues. These deactivated GameObjects will not be visible in the scene, and will not 
affect any execution within the scene. They can be turned on or off manually in the 
editor, or dynamically within a script. 


Class Hierarchy 

Using C 7 ^, scripts attached to entities may be structured into an inherited class 
structure. For example, an "Agent" class may hold basic attributes like "health" and 
methods like "doDamageQ" that all game entities will need. A similar approach may 
be taken for munitions or other non-entity objects which share a baseline of attributes 
and functionality. This structure is not strictly necessary, but it is helpful to reduce 
complexity by leveraging the OOP paradigm. 


Public and Private Variables 

Unity scripts may have both public and private variables and methods. Only public 
variables will appear in the Unity editor’s inspector window, and these variables can 
be monitored or changed at run time by the developer. As would be expected, only 
public variables and methods are accessible from outside the referenced script. Private 
variables can be accessed by pausing execution of the scene and visiting the paired 
development environment, as discussed in section 5.2.2. 


Custom Methods 

As with most other languages, custom methods can be built into a script. These 
methods will only be called when there is a reference to them from a MonoBehaviour 
method. For example, a custom "DieQ" method may be built into a generic "Agent" 
script. Then, when the Update() method is called each frame update cycle, some 
code may check to see if the agent’s health variable is less than or equal to 0. If it 
is, then it would call the Die() method, and it would execute accordingly, possibly 
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removing the agent from the scene unless specifically programmed to include a marker 
for where an entity died. 


5.2.2 Scripting Extensions 

Due to Unity’s massive popularity, major third party software companies have part¬ 
nered with Unity to integrate its development with existing integrated development 
environments. Microsoft’s Visual Studio (VS) was used for editing and examining 
scripts for this thesis, as it offers full integration with the Unity editor. 14 VS allows 
Unity developers to select scripts from within the Unity Editor, and they will open 
in VS. VS can then be "attached" to that specihc instance of the Unity Editor, 
and variables can be explored within the script at run-time. Breakpoints may also 
be identified in the script, allowing developers to automatically pause the execution 
of the script to examine variables and function execution. Unity is shipped with a 
Unity-specific integrated development environment (IDE) called MonoDevelop. Much 
of the same functionality is achieved with MonoDevelop; the decision of which IDE 
to use is largely a matter of preference. 


5.2.3 Scripting Without MonoBehaviours 

Unity allows developers to leverage the full breadth of C#, making it feasible to 
write scripts that do not use the MonoBehaviour structure. Although this is possible, 
the intention of a behavior prototyping environment is to enable quick and dynamic 
changes to behavioral logic, and the MonoBehaviour construct is not limiting in this 
regard. Thus, it is typically best to stick to inheriting from the MonoBehaviour in 
order to prevent confusion with other Unity developers and to avoid unnecessary 
complexity. 15 


14 Microsoft’s Visual Studio tools for Unity can be found here: https://www.visualstudio.com/en- 
us / features / unitytools-vs.aspx 

15 Behavior Designer, the commercial off the shelf asset discussed in more detail in chapter 6, does 
not inherit from the MonoBehaviour class. However, it does explicitly leverage many of the methods 
found in the MonoBehaviour. 
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5.3 Limitations of Unity 

Although Unity is an excellent game engine, it is not a one-stop-shop for game or 
simulation development. It is not intended for detailed 3D model construction, nor 
is it particularly well-suited for use as a high-fidelity analytic simulation tool. 

However, myriad tools exist for building models, including the free and open-source 
Blender project. 16 As with most game engines, Unity has specific file formats which it 
supports, and conversion methods do exist for most formats not directly supported. 1 ' 

As previously discussed, simulations executed in Unity are inherently time-step sim¬ 
ulations, which leads to imprecise determinations of event timing. This is entirely 
acceptable for most behavior prototyping. The frame-by-frame debugging methods 
previously discussed can be used to identify and accept these artifacts, which will not 
likely transfer to a discrete event simulation. 

Another complication with Unity is that the execution of scripts is somewhat arbi¬ 
trary. For example, if multiple scripts utilize the Update() method-which is called 
once per frame-the order of execution in which these scripts will be updated is some¬ 
what arbitrary and unpredictable. Although a specific order can be set for execution 
of scripts within a scene, this doesn’t necessarily solve the problem of determining 
the proper order of execution. 18 Consider a situation where an "agent" script with a 
"doDamageQ" method is attached to two soldiers engaged in battle. These soldiers 
have the same range of weapon, and fire at each other at the same time. Whether 
or not the order of execution is arbitrary or specified, the order will be incorrect 
because they both fired at the same time. This typically doesn’t matter for behavior 
prototyping, but it is a limitation that prevents Unity from being used as a full-scale 
discrete event simulation for analysis. It is important that the behavior developer be 
aware of this difference from the target simulation. 

However, for the non-analytic purposes of prototyping behaviors, Unity performs 
exactly as needed. The artifacts that we mention here will be eliminated when the 

16 Blender can be downloaded from the following website: https://www.blender.org/ 

1 7 Information on formats supported by Unity can be found here: 

http://docs.unity 3d.com/462/Documentation/Manual/3D-formats.html 

18 Details on specifying the script order of execution can be found here: 

http://docs.unity 3d.com/Manual/class-ScriptExecution.html 
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behavior is transferred to a discrete event simulation. 


5.4 Unity as a Surrogate Development Environment 

The use of Unity provides many of the ideal features of a behavior prototyping en¬ 
vironment discussed in Chapter 4. Unity revolves around 3D virtual environments, 
which helps behavior designers better understand the implications of the logical struc¬ 
ture of behaviors. It has a flexible, intuitive, and extensible user interface, allowing 
for customized applications for behavior prototyping. This dynamic development pro¬ 
totyping environment helps to build dynamic behaviors in a timely manner, ideally 
shortening the iteration cycles for development. 

5.4.1 Leveraging 3D Virtual Environments 

The built-in 3D virtual environment that exists in a game development environment 
such as Unity allows designers to test their agent behaviors in a wide variety of 
circumstances within a short period of time. For example, developers are able to 
quickly adjust the geometry of the buildings in a MOUT scene and see how that 
change affects the emergent behavior of a complex behavior tree. Sometimes, simply 
watching a behavior play out can provide insight as to that behavior’s room for 
improvement. 

For example, recall the GoToStore example provided in section 2.1.5, shown again 
here in Figure 5.3. This HTN appears logical, and simply reading output from a 
console in a simulation like COMBATXXI would not likely provide insight as to its 
shortcomings. However, consider a situation where the keys are in the house, and the 
agent is in the house, but the agent is not holding the keys. Following the logic in this 
HTN, the agent would then walk to the store, as seen in Figure 5.4. It would not be 
difficult to add another branch that checks to see if the key is within sight or within 
range, and proceeds to "pick up" the key if it is. Then the agent would get into the 
car and drive to the store. This may seem obvious, but isn’t always so without the 
use of at 3D environment. 

In this example, it is easy also to move the various objects around and see how it 
affects the execution logic. 
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Figure 5.3: An HTN to get to the store. 



Figure 5.4: The agent walks past the key within plain sight. 

5.4.2 Availability of Low-Cost Tools 

Unity’s large developer base lias led to a significant number of models and develop¬ 
ment tools which can be leveraged by any other developer. The Unity Asset store is 
the primary distributor of Unity assets. For example, there are over 130 assets for 
sale in the "Scripting and Artificial Intelligence" section of the Asset Store, ranging 
in price from $0 to $300. The overall market, however, extends beyond the formal 
Unity Asset Store. Many open source projects choose to distribute their tools either 
exclusively via other sites, such as GitHub, or via GitHub and the Asset Store. Re¬ 
gardless of the source, an aspiring behavior developer has many choices when looking 
for potential time-saving commercial-off-the-shelf solutions. This section provides an 
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overview of several assets that were considered for potential use in this thesis. 


Open Source Projects 

Open source projects should always be considered first, as they are free and offer 
the best opportunity for collaboration. Free and open source projects may be hosted 
on the Unity Asset store, but they are also found on popular open source websites 
such as sourceforge.net and github.com. Open source projects should always be ap¬ 
proached with caution, and properly vetted within formal security channels of the 
using organization. The primary benefit to open source code in the context of behav¬ 
ior prototyping is that the code is readable. This allows designers to examine exactly 
how certain tasks are executed, and change or extend them as needed. 

The first resource that was found is an open source project titled CHP: C# HTN- 
Planner, which is an HTN implementation written in specifically for Unity. How¬ 
ever, it was originally authored for Unity 4, and while promising, seemed to be limited 
to environments which do not require replanning, unlike C0MBATXX1. A more com¬ 
plete tool with more out of the box functionality was required. 19 

Another project which was discovered late in the production of this thesis is titled 
behaviac, which supports BTs, FSMs, and HTNs. This tool was not sufficiently 
evaluated, but could be used for future behavior prototyping. 20 


Unity Asset Store 

Three projects were pursued that exist or have existed within the Unity Asset Store: 
Stagpoint’s HTN Planner, Rival Theory’s RAIN AI, and Opsive’s Behavior Designer. 

A development company called Stagpoint had produced a now unsupported HTN 
implementation within Unity. The description and online demos show an HTN im¬ 
plementation which appear similar to the HTNs in COMBATXXI. 21 Unfortunately, 
it was removed from the Asset Store prior to the start of this thesis, and efforts to 
contact the development team were unsuccessful. 

19 This HTN implementation can be found here: https://sourceforge.net/projects/chpplanner/ 

20 Behaviac can be found here: https://github.com/TencentOpen/behaviac 
21 Stagpoint’s HTN Planner description can be found here: http://stagpoint.com/planner/ 
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Rival Theory’s RAIN Artificial Intelligence platform is free in the Unity Asset Store. 
As a result, it is widely used and there are plenty of user-created tutorials. RAIN has 
a unique implementation of BTs that is relatively easy to use and potentially useful 
for creating HTNs in Unity. 

However, the source code is proprietary, and users doesn’t have a way to know how 
a certain automated behavior works under the hood. For example, RAIN allows for 
easy creation of a visual sensor that is based on a line of sight (LOS) algorithm, but 
the user can only guess if the determination is based on a raycast to the center mass of 
the target, or some other point. Without specific knowledge of how a given algorithm 
works, some level of confidence in the ability to translate that code to COMBATXXI 
HTNs is lost. Furthermore, there is no current support for extending RAIN AI, so 
custom behaviors meant to prototype COMBATXXI HTNs could not be built. 

The final tool which was explored for use as a prototyping tool was Opsive’s Behavior 
Designer. 22 Behavior Designer costs $105 per seat license in the Unity Asset Store, 
including all three add-on packs which were experimented with for this thesis. While 
not free, this cost is minimal in comparison to the in-house development costs of 
building a similar tool. Furthermore, the online forums and responsive support team 
ease the use and troubleshooting of Behavior Designer behavior trees. 

Behavior Designer provides an extension of the LInity editor which allows designers 
to build custom behavior trees in a separate editor window and watch them execute 
at run-time. A simple example of a behavior tree in Behavior Designer is shown in 
Figure 5.5. Each node in the tree has an associated script, and each script can easily 
be opened in the users IDE of choice. Scripts allow the user to examine exactly how 
a given task is being executed, and to improve upon that task if they desire. New 
custom tasks can also be built and integrated into the Behavior Designer development 
environment. It is the flexible and extensible nature of Behavior Designer which led 
to its selection for use in this thesis. 


2 2 An overview and documentation of Opsive’s Behavior Designer can be found here: 
http: / / www.opsive.com/assets/BehaviorDesigner / 
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Figure 5.5: A behavior tree which sends an event after detecting an enemy. 


5.4.3 Proceeding with Unity and Behavior Designer 

Although no HTN implementation exists for Unity which is similar enough to the 
COMBATXXI HTNs to provide one-to-one mapping of behaviors, the combination 
of Unity and Behavior Designer provides behavior designers a sufficiently flexible 
environment to experiment with behaviors that are transferable to COMBATXXI. 
Chapter 6 provides a detailed description of the development environment that was 
built to prototype behaviors for this thesis. 
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CHAPTER 6: 

Using Behavior Trees in Unity to Emulate HTNs 


This chapter provides additional examples of behavior trees that are used to emu¬ 
late COMBATXXI HTNs. While the basic example of the GoToStore behavior was 
demonstrated in section 2.2.2, another behavior from a well-known COMBATXXI 
HTN tutorial is used here to demonstrate a more complex behavior. Lastly, attempts 
made at prototyping new behaviors within Unity for translation to COMBATXXI 
are detailed. 


6.1 Utilizing the Prototyping Environment 

Details on the construction and layout of the prototyping environment have been 
encapsulated in Appendix A. This section discusses the methodologies and concepts 
used for building the prototyping environment, and following sections will discuss the 
theory and practice of structuring behavior trees to emulate COMBATXXI HTNs. 


6.1.1 A Model of a Model 

Although some commercial off the shelf (COTS) tools were used, the majority of this 
prototyping environment was built from empty scenes within Unity. Building it was 
a piecemeal process; objects were built as needed during the prototyping process. 
The tools needed to replicate the GoToStore behavior, for example, were smaller and 
less robust than those needed to prototype the more complex behaviors that actually 
transfer to COMBATXXI. However, LInity’s ease of use can be a siren’s call to develop 
interesting tools or behaviors that might not actually transfer to the target system. 
After much trial and error in transferring behaviors to COMBATXXI, the "model of 
a model" approach became the cornerstone of the methodology used in this thesis. 
This means that no object or behavior should be built in the prototyping environment 
which will not transfer to the target simulation. In general, designers should be 
cognizant of the representational space of the target simulation when constructing 
the prototyping environment. 
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Not all aspects of the target simulation can or should be emulated in Unity, but that 
doesn’t mean that an abstraction of an abstraction isn’t still useful. For example, 
the basic action of engaging an enemy target in COMBATXXI is a complex process. 
It involves target selection, weaponeering, 23 starting the engagement, operating the 
weapon, creating the munition object, determining the munition accuracy and fly out, 
assessing the impact, and assessing the damage [39]. An adequate target engagement 
process in Unity might include target selection, raycasting to verify line of sight, and 
assessing damage. In this case, the behavior designer must be aware of the nuance of 
the Engage Functional Module in COMBATXXI and how it might affect the validity 
of the behavior. Behavior designers should emulate the least amount of detail as is 
necessary for their particular behavior. It is easy to get bogged down in wanting to 
perfectly emulate the functional module as it exists in COMBATXXI, but doing so 
would defeat the purpose of leveraging the time savings that are inherent in using a 
surrogate development environment for prototyping purposes. 

Another example of when less resolution in Unity is appropriate is the observation 
process. As with many "off the shelf" tools, Behavior Designer has built-in tasks for 
checking a field of view out to a given range. The task may then return whether or 
not an enemy is within the field of view. In COMBATXXI, the observation process is 
driven by a held of view and a held of regard. 24 While the ability to model a held of 
regard may be necessary for some high-fidelity military simulations, setting the held 
of regard equal to the held of view for the purpose of prototyping behaviors in Unity 
is completely acceptable. 

Whereas an entity in COMBATXXI may take some time to "find" an enemy in its 
held of view, the detection of a given object in the held of view of an entity in Unity 
would happen within one "tick", or the time it takes the computer to iterate through 
the graphics pipeline-typically sixty times per second. This means that the time to 
detect an entity in COMBATXXI will have far more variability than in Unity. If 
we are modeling the observation process, this is not acceptable. However, if we are 
modeling a given reaction to an observation event, this abstraction probably will be 


23 Weaponeering tables are used in COMBATXXI in order to select the best available weapon 
and ammunition for a given target at a given range. 

24 Field of view is the area within the field of regard that the entity is currently focused on. 
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acceptable. It is the duty of the behavior designer to use good judgment on what 
level of detail to model. Typically, it will be the same behavior designer who reaps 
the rewards or pays the punishment of wasted time if this is not done judiciously. 


6.2 Emulating COMBATXXI HTNs with Behavior 
Trees 

This section combines the fundamentals of HTNs detailed in section 2.1.3 with the 
lessons learned in building behavior trees for use as HTNs. Using the "model of a 
model" concept, behavior trees that do not translate to COMBATXXI are ultimately 
useless for our purposes. Developing a baseline of understanding for how behavior tree 
components translate to HTN components will help ensure developers build behaviors 
which are more likely to transfer smoothly to COMBATXXI. 


6.2.1 Additional Considerations Regarding the Anatomy of an 
HTN 

Most components discussed in this section were identified in the "Anatomy of an 
HTN" section, section 2.1.3. However, some additional concepts are included here 
which were found to be important aspects of HTN design. 


Initializing the HTN 

Variables are managed in several ways within an HTN, but they are almost always 
gathered when the HTN is initialized. This can be modeled in Behavior Designer’s 
behavior trees explicitly by the use of an "initiation" branch of a tree, or implicitly by 
the use of the "shared variables" tab, which allows behavior trees to store variables 
in one place for use throughout the tree. These variables can be provided as input if 
the tree is instantiated via a behavior tree reference (see section 6.2.1), procedurally 
from within the tree itself, or manually by a designer entering values. There is no 
incorrect way to do this, so long as the designer knows how the values will translate 
to the initialization of the HTN. 
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The isCommander Pattern 

When architecting behaviors for COMBATXXI, it is often easier to give a behavior 
to all entities in a unit, and screen the execution of that behavior or a portion of that 
behavior with the "isCommander" pattern. This pattern simply uses a constraint 
node to check whether or not an entity is the commander of his unit at a given 
echelon. This can be emulated by including a public Boolean "isCommander" to the 
AgentState class script, as detailed in Appendix A. 

Modeling Interrupt Goal Tasks 

The intent behind interrupt goal tasks in COMBATXXI HTNs is to prevent the 
planner from planning too far into the future and to represent tasks that take some 
time complete [18]. The HTN pauses at this node, and then begins again once the 
interrupt goal task is complete and an appropriate replan event occurs. This helps 
to prevent too much computational waste by planning too far into the future when 
the world state will likely necessitate a change to the plan after the execution of the 
interrupt goal task. 

This is not easily replicated in behavior trees, because they do not plan. Therefore, 
an interrupt goal task as modeled in a behavior tree is no different than any other 
task which takes some time to execute. However, it is important for designers to keep 
in mind logical stopping points for the HTN to pause. This is usually the last node in 
a sub-branch of a tree, if that branch is executed as a child of a sequence or selector 
node. In the GoToStore example, each of the movement nodes demonstrates how to 
model interrupt goal tasks, as seen in Figure 2.9 on page 22. 


Replanning with Model or Goal Tracker Events 

In order to force a behavior tree to emulate replanning as an HTN would, it needs 
to start again from the top. This is accomplished in one of several ways. Either a 
node which is part of a sequence node fails, or a node which is part of a selector node 
succeeds. This returns execution up to the next highest node. If this happens to 
send execution all the way to the top of the tree, and the tree is set to repeat upon 
completion, then this will effectively emulate a replan trigger in a COMBATXXI 
HTN. Alternatively, a repeater node may be placed at the top of the tree, under 
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the entry node, for an explicit visual reminder that the tree is set to repeat. This is 
demonstrated in the GoToStore example in Figure 6.1 



Figure 6.1: Placing a repeater node at the top of the tree or selecting the 
"Restart when complete" box will allow for emulation of a replan event. 


Goal Nodes in Behavior Trees 

The use of repeater nodes in a behavior tree means that it will not stop unless 
explicitly told to do so. However, the "stop behavior tree" node works nicely for this. 
Although the "stop behavior tree" node does not itself represent the goal, it is a way 
for the designer to implicitly mark that the goal has been achieved. For example, 
if a soldier’s task is simply to observe and report, then after he has reported visual 
contact of the enemy, the goal has been achieved. This example is demonstrated in 
Figure 5.5. 

Adding COMBATXXI-like Goals in Behavior Trees 

One of the more common ways to add a goal to an entity in COMBATXX1 is to use 
the addGoal() method. This allows a designer to add an HTN to any entity, and to 
enter the parameters directly from within this function call. There are two main ways 
this can be emulated in Behavior Designer’s behavior trees. First, a designer can use 
the behavior tree reference node. At run time, this node imports a common behavior 
tree from the Unity project as a new branch where that node had existed. Variables 
can be synchronized and passed to the tree, or the tree can rely on existing shared 
variables from within the parent tree. 

The second way to emulate the addGoal() function call is to use the start behavior 
tree node. This node requires that the behavior tree to be used already exists, and it 
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should have a unique "group" number associated with it. This behavior tree should 
be turned off initially, but attached to the entity that the designer expects to need it 
later. 

Combining these concepts allows designers to prototype basic HTNs within Unity. 
There are likely other ways to accomplish this, and no claim is made that this method 
is optimal. However, subsequent examples will help to demonstrate that it does work. 


6.3 Bounding Overwatch 

The Bounding Overwatch tutorial is used to introduce new HTN designers to the nec¬ 
essary concepts and user interfaces. 25 This section describes the Bounding Overwatch 
scenario as recreated in Unity. 

The Agents 

There are 8 soldiers, comprising a single infantry squad. There is a squad leader and 
an assistant squad leader. Otherwise, all soldiers are identical, and are instances of 
the Soldier prefab as identified in Appendix A. 

The Environment 

This scenario is built on a flat plane as terrain, and there are six buildings along a 
simulated road. The plane includes a navigation mesh, as is required in Unity in order 
to allow the NavMeshAgent component of the prefab instances to move about the 
scene easily. The road is marked by the path between the squad’s starting location 
and the last way point. 

The Behavior Goal 

This behavior is used in COMBATXXI to dynamically determine whether a squad 
should split up and move in bounding overwatch or stick together and move in a 
single formation. Bounding overwatch is similar to a leapfrog scheme of maneuver, 
where one unit provides cover while the other moves. Each unit alternates movement 

25 The Bounding Overwatch tutorial can be found at the NPS 
"Tutorials and Examples" page on the WSMR wiki website: 
https: //cxxi.wsmr.army.mil/confluence/display/NPS/Tutorials+and+Examples 
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until the maneuver is complete. The determination between whether or not the squad 
will move in bounding overwatch is made based on the number of buildings along a 
road. This number is retrieved in COMBATXXI using a Python function that simply 
returns the number of buildings along a road. In order to emulate this in Unity, a 
custom task was built which counts the number of GameObjects with a certain tag 
within a certain range and sector relative to the using entity. Once this number 
is retrieved and stored, the behavior tree shown in Figure 6.2 is able to direct the 
execution flow to the appropriate branch. If there are four or more buildings along 
the road, the flow of execution is directed to the left side of the tree, which starts the 
bounding overwatch behavior for the squad leader and assistant squad leader. 



Figure 6.2: This behavior tree enables the squad leader to count the number 
of buildings as a means of determining whether or not to move by bounding 
overwatch or by a squad wedge formation. The node labeled "Compare 
Shared Int" in the bottom left, used as the leftmost node in a set of sequence 
child nodes, emulates a basic constraint node as in COMBATXXI HTNs. The 
two "Start Behavior Tree" nodes initiate the behavior trees for the squad 
leader and assistant squad leader. These trees are detailed in Figure 6.3. 

If the squad moves in bounding overwatch, execution moves to the two trees started 
by the "Start Behavior Tree" nodes that are seen in the bottom of Figure 6.2. The 
squad leader’s subsequent behavior is similar to the assistant squad leader’s behavior, 
as shown on the left and right side of Figure 6.3, respectively. 

If the number of buildings returned is less than or equal to three, then the squad will 
simply move in a single wedge down the middle of the road to the final way point. 

This scenario, built in Unity, enables developers to quickly change the orientation 
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Figure 6.3: These two behavior trees are identical, except for their variables. 
The behavior trees for the squad leader and the assistant squad leader (i.e. 
second-in-command, or SIC) to take their respective halves of the squad 
along the appropriate side of the road using the bounding overwatch tactic. 
This provides an example of the flexibility of behavior trees. 


and number of buildings, the balance in the number of buildings on each side of the 
road, as well as the size and movement route of the squad. This rapid testing can 
help to identify weak points in the behavior logic and find the most appropriate way 
to make the behavior more robust to changes in the simulation world state. 


6.4 Prototyping New Behaviors 

The Platoon Security Patrol is a COMBATXXI scenario that has been used in the 
past by OAD, but for which no current implementation exists that uses HTNs. For 
this reason, it was selected as a target for potential revision as part of this thesis. 

6.4.1 The Scenario in COMBATXXI 

The Platoon Security Patrol scenario involves a platoon of Marines that conduct 
a mounted patrol, moving south to north along a main supply route, or road. At 
some point during their patrol, one of the vehicles strikes an improvised explosive 
device (IED), and the platoon conies under small arms fire from enemy positions 
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to the west. The Marines then call for supporting fire in the form of IDF from a 
nearby 60mm mortar team and close air support (CAS) from nearby Cobra attack 
helicopters. The platoon then maneuvers to the north and flanks the enemy positions. 
The scenario, as it exists in COMBATXXI, includes 60 blue forces and 77 red forces. 



Figure 6.4: An overview of the Platoon Security Patrol provided by OAD. 

In order to simplify the scenario, the blue forces were reduced to 33, and the red forces 
were reduced to 30. Additionally, the CAS assets were removed, so the Marines in 
the reduced scenario would have to rely only on the on the IDF assets. This sim¬ 
plification allowed for increased focus to be placed on the development of behaviors. 
In keeping with the qualities discussed in section 4.2.1, the scenario was analyzed 
and distilled into modular actions which needed to be modeled in the prototyping 
environment. That distillation is specified in the following two subsections, divided 
by agent affiliation (e.g. red or blue) and ordered chronologically. 


Blue Agent Actions 

1. Load dismounted soldiers into their assigned vehicles. 

2. Order vehicles to patrol along main route. 

3. React to IED-induced catastrophic kill of the lead vehicle by taking a pre¬ 
specified secondary route which will be used to flank the enemy position. 

4. Submit call for fire to nearby forward observer. 

5. Call to cease fire when in position to assault the enemy. 

6. Dismount troops when within appropriate range of the enemy. 
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7. Conduct frontal assault through enemy positions. 

Red Agent Actions 

1. Engage enemy with small arms fire when IED detonates. 

2. Take cover when enemy fire support assets begin firing on friendly positions. 

3. Move to secondary fighting positions when force strength is reduced to 60%. 

4. Defend secondary fighting positions to the death. 

6.4.2 Platoon Security Patrol in Unity 

After the scenario was reduced and the agent actions outlined, development began in 
Unity. A similar scene was constructed, and a subset of agents were created to build 
a minimal working example of the scheme of maneuver depicted in Figure 6.4. 

Once the tasks were specified, development began in Unity, without periodically up¬ 
dating the corresponding behavior in COMBATXXI. This turned out to be problem¬ 
atic, as is discussed in section 7.1.1. 

The Agents 

There are a total of Eve entities in this scenario. Three are blue soldiers, one is a blue 
vehicle, and the last entity is a red soldier. 

The Environment 

The environment is similar to the Unity version of the bounding overwatch scene, with 
the exception of being larger and allowing for more maneuver possibilities. There is a 
graphically depicted road, and buildings based on the prefabs specified in Appendix 
A. The scene is depicted in Figure 6.6. 

The Behavior Goal 

The behavior built in this scene was used as a demonstration of the capabilities 
of the prototyping environment, and does not include all components of the actual 
platoon security patrol scenario. The blue soldiers begin by loading into the vehicle 
and patrolling north. They then dismount just short of reaching an enemy IDF zone 
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where mortar rounds are actively falling. Once they have dismounted, they follow a 
scripted route by way of several way points in order to conduct a frontal assault on 
the enemy position. The corresponding behavior tree, which belongs to the squad 
leader, is depicted in Figure 6.5. This point in the scenario is depicted in Figure 6.6. 


BlueSoldier_AgentState - BehaviorController 




Figure 6.5: This linear behavior tree is simply a starting point for developing 
original behaviors for the Platoon Security Patrol. Note that 
for "vehicle." 
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Figure 6.6: A Unity scene built to prototype the Platoon Security Patrol 
scenario behaviors 


Meanwhile, the enemy red soldier is calling for fire on a location near the blue patrol 
route. This behavior was built as a custom task called "call for fire", which places 
a red "X" at a center point and initiates 60nnn mortar round detonations within a 
specified box around that "X". This behavior and its associated prefabs are specified 
in more detail in section A.2.2 on page 93. 
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All soldiers are able to engage their enemies with direct fire once they are within 
range, and once either all blue soldiers are dead or all red soldiers are dead, the 
scenario is over. There is an inactive IED in the scene as well, which could be used 
for further prototyping of reactions to IEDs. 

6.4.3 Translating to COMBATXXI 

After a baseline of functionality was developed in Unity, development efforts were 
shifted to the COMBATXXI scenario. This proved to be the most difficult part of the 
process, for two main reasons. First, the tasks in Unity do not map to COMBATXXI 
except on a conceptual level. Second, the development environments both have their 
own nuance that tend to thwart novice developers. For example, a mounted patrol 
seemed to not be moving in the COMBATXXI scenario, and significant development 
time was spent trying to End a reason for this within the HTN. The problem was 
resolved by turning off the modeling of "detailed ground formation." 

However, with assistance from experienced COMBATXXI developers, a partially com¬ 
plete scenario was developed that does implement indirect fires, an IED detonation, 
and the use of a secondary route as a reaction to the IED detonation. No red soldier 
actions were programmed, and the blue soldiers stop short of dismounting to conduct 
the frontal assault. This scenario in action is depicted in Figure 6.7. 
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Figure 6.7: In the Platoon Security Patrol scenario, a three vehicle convoy 
patrols north along the "Main Route" until the lead vehicle is killed by an 
IED. The remaining two vehicles then take the "Secondary Route," and call 
for fire support from the nearby forward observer. The flanking behavior, 
frontal assault via the "Assault Routes," and the red soldier behaviors were 
left as future work. 
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CHAPTER 7: 

Recommendations and Conclusions 


Developing behaviors for COMBATXXI scenarios is the current bottleneck for de¬ 
velopment of scenarios for analysis. Finding ways to more efficiently build and test 
behaviors will continue to be the lowest hanging fruit for those seeking to improve 
the scenario development process. Using visually intuitive tools such as game engines 
is a previously unexplored methodology which may provide benefit inherent in work¬ 
ing with a 3D environment. This thesis involved building a behavior authoring and 
preliminary testing environment using the Unity3D game engine. The differences in 
development environments between the prototyping environment and the target sim¬ 
ulation created unforeseen barriers. Specific work flows were explored out of necessity, 
and suggestions on their utility are provided in this chapter. Building upon the al¬ 
ready robust COMBATXXI development environment is explored as potential future 
work. Additionally, there are other ways in which game engines could be used to im¬ 
prove the scenario development process, such as leveraging existing interoperability 
standards to use the game engine as a replay tool for behavior testing. 


7.1 Recommendations 

Developing behaviors in a surrogate environment has many potential benefits. It pre¬ 
vents developers from being tempted or directed into including prototyped behaviors 
in production-level simulation environments. It also enables developers to leverage 
gap-filling technologies from outside the target simulation, or to evade the whole- 
model complexity that may exist in the target simulation. However, developing in 
a surrogate environment requires that the developer be intimately familiar with the 
capabilities and limitations of both environments. Additionally, if there is not an 
automated transfer process, then the developer must manually map code from the 
surrogate environment over to the target simulation. While this may prove worth 
while, it is worth considering up front the approach that will best fit the surrogate 
and target simulation pair as well as the experience of the developer. 
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7.1.1 Choosing a Development Work Flow 

Developers have a choice when working with a surrogate environment as to how 
frequently they transfer prototyped behaviors to the target simulation. There are 
benefits and detractors at both extremes. Developing an entire scenario in the surro¬ 
gate environment before visiting the target simulation allows the developer to immerse 
their thoughts in the code and model nuances of the surrogate environment. Contrast¬ 
ingly, frequently visiting the primary development environment helps to ensure that 
the behaviors built in the surrogate environment are feasible in the target simulation. 
Both approaches are potentially valid, and are discussed in more detail below. 


Developing in Series 

When working in a surrogate environment that is significantly different from the pri¬ 
mary development environment, it is easy to become immersed in that environment. 
This is particularly true if the surrogate environment is easier to use. It is a natural 
inclination to not want to go back to programming in the more complex environ¬ 
ment with a more complicated model, and this represents a possible pitfall for the 
inexperience developer. 

However, for an experienced programmer, this may be the most appropriate approach. 
If they are confident that what they have prototyped will transfer, the developer could 
develop whole behaviors or even entire scenarios in the surrogate environment before 
turning back to the target simulation. This process is visualized as a UML sequence 
diagram shown in Figure 7.1. This method is similar to the waterfall method of 
software development, except that it is typically one person conducting each phase 
of development [40, p. 329]. 

The inherent risk with this approach is that development efforts in the surrogate 
environment that do not transfer to the target simulation are wasted. This waste 
ties back to the "model of a model" concept which was discussed in section 6.1.1. If 
a non-mapping element of a behavior exists fairly high up in the behavior tree, for 
example, the entire behavior tree may need to be refactored. This was the approach 
initially taken for this thesis. 
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Developing in 
Unity 


Transferring to 
COMBATXXI 


Testing in 
COMBATXXI 


Behavior Trees 


Complete Scenario 

^ 1 

Figure 7.1: Developers may choose to build entire scenarios in the surrogate 
environment first, incurring the risk that some behaviors won't transfer. 
Ideally, this process is conducted once per scenario. 



Developing with an Iterative Approach 

Alternatively, developers may prototype individual behaviors or even small portions 
of individual behaviors, and immediately transfer them to the primary development 
environment. This process is visualized as a UML sequence diagram shown in Figure 
7.2. This method helps to ensure transferability, and prevents significant development 
time in the surrogate environment from being wasted. This can be viewed as an 
implementation of the spiral development process [41, p. 64], 



Figure 7.2: Developers may choose to frequently transfer existing behaviors 
to the target simulation in order to prevent excessive refactoring. This pro¬ 
cess is repeated as many times as is necessary, depending on the number of 
behaviors required. 


For an experienced developer, this method of repeatedly visiting the primary devel¬ 
opment environment may be wasteful. Going back and forth between two differ- 
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ent domains that likely use different programming languages and require familiarity 
with various model specifics can be confusing. For example, developers in Unity 
should be familiar with the MonoBehaviour lifecycle depicted in Appendix B, while 
COMBATXXI developers need to thoroughly understand discrete event simulations. 
Even an experienced developer will take some time to move local variables, frequently 
used functions, and scenarios specifics into their short-term working memory. For in¬ 
experienced developers who do not have a thorough understanding of what is possible 
in COMBATXXI, this may be the only feasible approach. 

These two methodologies exist on opposite ends of the spectrum. On one side, an 
ideal developer who does not make mistakes could build an entire scenario in the 
prototyping environment before transferring anything to the target simulation. On 
the other end of the spectrum, a complete beginner in the target simulation or the 
prototyping environment may have to use much shorter iterations to ensure transfer- 
ability. However, the reality is that any single developer will likely evolve at some 
point between these two extremes. For example, our beginner developer may begin to 
gain familiarity with function calls, modules, or libraries from the target simulation 
that map to the functionality of the prototyping environment. This will allow the 
beginner developer to spend less time bouncing back and forth between environments. 
Contrastingly, our experienced developer may undertake a project which requires him 
to learn a new portion of the target simulation with which he was not yet familiar. In 
this case, he will need to return to the target simulation more often when prototyping, 
until he becomes expertly familiar with said portion of the target simulation. 


7.1.2 Building an HTN Implementation in Unity 

A significant portion of the cognitive effort required to model HTNs in Unity is 
dedicated to translating between the semantics of behavior trees and the semantics 
of HTNs. Behavior Designer, the Unity asset used for this thesis, was built by a third 
party development group. It may be feasible to develop an HTN implementation in 
Unity which more directly translates to HTNs in COMBATXXI. Certain inherent 
limitations, such as the discrete event nature of COMBATXXI and the time-stepped 
nature of game engines, would difficulty in creating a direct translation. However, if 
the semantic structures of the behaviors are the same, then this would be less of a 
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concern. Additionally, implementing discrete event simulations in Unity is not outside 
the realm of possibility, as indicated by current work at the MOVES Institute [42], 


7.2 Future Work 

There are other ways in which the development process may be improved. COMBAT- 
XXI and its associated development suite may be extended to enable faster authoring, 
testing, and debugging. Game engines and other visualization methods may be ex¬ 
plored to provide a visual component to the testing and debugging processes within 
the current COMBATXXI development cycle. Behavior standards across the DoD 
M&S community could be developed in order to ensure better transferability of be¬ 
haviors between existing simulations. Additionally, Unity or other game engines could 
be explored as potential platforms for analytic simulations. 


7.2.1 Extending COMBATXXI 

Section 3.3.1 suggests options for extending COMBATXXI as an alternative to the 
prototyping methodology suggested in this thesis. Although this thesis did not explore 
any specihc in-roads for extending the COMBATXXI development environment, the 
difficulties experienced when transferring behaviors from the Unity to COMBATXXI 
demonstrated that prototyping the behaviors within the COMBATXXI framework 
could be preferable. Specifically, the main benefits of using Unity are the 3D vir¬ 
tual environment and the ability to debug behaviors at run-time. If either of these 
were achievable within the COMBATXXI framework, that would likely be a more 
preferable solution to prototyping in a surrogate environment. 

7.2.2 Using Unity3D as a COMBATXXI Playback Tool 

Several tools exist that allow Unity developers to integrate DIS functionality into their 
simulations, provided they are willing and able to build out the DIS enumerations 
that would be sent from COMBATXXI. 26,2 ' COMBATXXI can provide DIS output 

26 Calytrix’s LVC Game, built for Unity 4.x, provides a commercial off the shelf solution, and can 
be found here: http://www2.calytrix.com/products/lvcgame/introduction/ 

2 'Karl Jones’ DIS-Unity provides an open source solution, also for Unity 4.x, and can be found 
here: https://sourceforge.net/projects/unitydis/ 
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files which could be played back over a network and visualized in Unity. This would 
allow developers to visualize their scenario output in a way that is not currently 
possible. This method would have to overcome the typical array of interoperability 
concerns, such as DIS enumeration configuration and terrain matching. However, the 
payoff could well be worth the effort. 

7.2.3 Unity3D as an Analytic Tool 

When developing scenes which had stochastic processes in them, such as the platoon 
security patrol’s indirect fires, it became evident that Unity could be used as a tool for 
analysis. Although there are known problems with the use of time-stepped simulations 
for analytic purposes, there are still analytic simulations that are time-stepped. For 
example, Map Aware Non-uniform Automata (MANA) uses time arbitrary steps to 
mark simulation time. 28 NetLogo, a popular tool for building large scale agent-based 
simulations which have been used to model complex social systems, uses "ticks" 
to mark its time. If Unity were used for analysis under the same precepts which 
guide the use of other time-stepped simulations, then it would be equally valid. This 
thesis focuses on prototyping behaviors for use in a Discrete Event Simulation (DES). 
Perhaps the use of Unity as a tool for analysis would be more appropriate for simple, 
agent-based simulations. These simulations often accept the implicit issues of a time- 
step simulation, as evidenced by the continued use of MANA and NetLogo. In these 
simulations, developers aim to leverage the first theme of complexity research, as 
identified by Ilachinski, that "surface complexity can emerge out of a deep simplicity" 
[3, p. 6], 

7.2.4 Standardization of Behavior Specification 

One way in which M&S professionals could more reliably leverage behavior devel¬ 
opment surrogate environments would be to develop a standardized framework for 
behaviors. Simulation Interoperability Standards Organization (SISO) is the "sim¬ 
ulation interoperability through reuse and standards" [43]. SISO currently has no 
such standard, likely because behaviors are often authored in different programming 

28 Map Aware Non-uniform Automata (MANA) is a simulation tool developed by the New Zealand 
Defence Technology Agency (DTA). 
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languages and using different structures, as discussed in Chapter 2. The same logic 
that guides good transferability between simulations as embodied in other SISO stan¬ 
dards could also guide behavior development in surrogate development environments 
to target simulations, ensuring better transferability. 

7.2.5 Conclusions 

Although the complete goal of this thesis was not accomplished within the speci¬ 
fied time frame, a workable environment for building COMBATXXI HTNs in Unity 
was completed and the basic premise that such an approach is viable was validated. 
That environment was used to emulate existing HTNs and to prototype new HTNs. 
However, there is still much more that can be done to create better behaviors for 
use in military simulations. These improvements may involve additional work with 
game engines as surrogate environments or visualization tools, or they may involve 
extending the current suit of COMBATXXI development tools. 

Regardless, agent behaviors will continue to be a primary enabler of entity-level com¬ 
bat simulations, and developers’ ability to build reusable and robust behaviors in a 
timely manner will play a key role in determining the timeliness with which the DoD 
analytic community can provide M&S-enabled insight to decision makers. 
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APPENDIX A: 

The Prototyping Environment 


This appendix specifies the prototyping environment which has been built in pursuit 
of the completion of this thesis. Unity3D version 5.3.1 and Opsive’s Behavior Designer 
version 1.5.5 were used in its construction. 

To explore this tool, a user must have purchased Behavior Designer and have down¬ 
loaded the appropriate packages specified here in order to use the custom behavior 
tree tasks which were built for this project. Then, download the Unity project at 
the linked repository. 29 There is a complete repository maintained by the MOVES 
Institute which holds the fully functional Unity project as well as the appropriate seat 
licenses. Subsequent theses can leverage this repository. 30 The user should then set 
aside the files titled "Behavior Designer Custom Tasks", so that it does not interfere 
with the installation of Behavior Designer. Once Behavior Designer is installed, locate 
the appropriate subfolder within the file hierarchy, and insert the base folder titled 
"CustomCXXI" tasks there. This will ensure proper integration with the Behavior 
Designer architecture. 


A.l Behavior Designer Behaviors 

Although Behavior Designer has a good deal of out-of-the-box functionality, some 
extensions were made specifically to allow for behaviors which are more similar to 
COMBATXXI behaviors. This section specifies how those behaviors are organized 
and what their intended purposes are. 

A. 1.1 Custom Tasks 

In order to keep the integration of these custom tasks with the larger Behavior De¬ 
signer system as simple as possible, all tasks were kept in a single folder. This is not 

29 The following bitbucket repository holds the partial project, with all proprietary Behavior 
Designer components removed: https://bitbucket.org/nps_millerthesis/unity_htn_prototype 
90 To request access for and download the full Unity project from within the nps.edu domain: 
http: //pogy.ern.nps.edu/gitbucket / dmmiller / unity-htn-prototyping 
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strictly necessary, but doing otherwise would make integration of these scripts need¬ 
lessly difficult for subsequent users. However, some tasks may inherit functionality 
from other tasks, and therefore will not all show up in one place within the Behavior 
Designer tool, as shown in Figure A.l. 



Figure A.l: Type "custom" into the search bar of the task tab within the 
Behavior Designer editor to find all uploaded tasks built for this thesis. 

Each task is contained within a script, and documentation is included for each 
task within the script. 

A. 1.2 Custom Behavior Trees 

Custom behavior trees were used whenever possible, as they allow for modularity and 
code reuse. Repetitive compound tasks, such as getting in a vehicle, can be wrapped 
into a custom behavior tree and used from within a larger behavior tree to simplify 
the behavior design process. 

The behaviors which were built into behavior trees for this project are as follows: 

EmbarkVic 

This behavior, shown in Figure A.2, is attached to a unit leader who may or may not 
have subordinate agents within the unit, ft must be given a GameObject which 
represents the parent vehicle. This emulates the Transport Group concept from 








COMBATXXI. The squad leader then moves the squad to the vehicle in a default 
wedge formation, and loads them into the vehicle by convolving all dismounts to 
the center point of the parent vehicle. This is accomplished in Unity by moving the 
position of each soldier to the center point of the vehicle. 



Figure A.2: The "EmbarkVic" behavior tree loads a squad into a vehicle. 


EngageUntilDeadOrGone 

This behavior, shown in Figure A.3, can be given to all entities in order to allow 
them to engage enemies within a certain range. It simply detects objects within line 
of sight, checks to see if any are enemies, and targets the nearest enemy. The enemy 
type is set to a default, as is the range, but both values can be overridden from the 
parent tree. If the enemy moves out of range, the behavior is reset. 



Figure A.3: The "EngageUntilDeadorGone" behavior engages the nearest 
enemy within range. 
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DetectCloseWithEngage 

This behavior, shown in Figure A.4, was the behavior originally built to emulate en¬ 
gagement in COMBATXXI. It would be added to every combatant in a scene, and 
turned on or off as needed. However, the "pursuit" portion of this behavior is not 
similar to engagement in COMBATXXI, which is separate from movement. This led 
to the development of the similar "EngageUntilDeadorGone" behavior, which does 
not include a pursuit element. However, this behavior was kept in this collection be¬ 
cause it serves as an example of modularity. It is triggered by line of sight detection 
of an enemy, after which point the two subsequent behavior trees are run in paral¬ 
lel. This allows developers to change one of the subordinate trees-called "behavior 
tree references" in Behavior Designer-and simply plug it in here without changing 
the structure of the parent tree. The use of a "behavior tree reference" in Behavior 
Designer is most similar to the use of the AddGoalQ method in the COMBATXXI im¬ 
plementation of HTNs, although this Behavior Designer method is limited to adding 
a goal to the tree’s owning agent. 



Figure A.4: The "DetectCloseWithEngage" behavior results in a search and 
destroy type of behavior after detecting an enemy. 








MoveToEngagementRange 

This behavior, depicted in Figure A.5, closes the gap between an agent’s detection 
range and its engagement range. For example, it may be capable of detecting an 
agent at 600 meters, but needs to move to within 500 meters to engage. This behavior 
allows for modeling of that behavior. Ultimately, this is not the default behavior in 
COMBATXXI, so this behavior was not included in any of the final scenes. 



Figure A.5: The "MoveToEngage" behavior moves the agent to within en¬ 
gagement range. 


Destroy WithinRange 

This behavior, depicted in Figure A.6, is the "engage" portion of the "DetectClose- 
WithEngage" behavior. It engages the nearest target using a public method from a 
script attached to the target that reduces the target agent’s health. 

It is important, when using modular behaviors, to understand their originally in¬ 
tended use and scope. Opening the custom behavior tree and examining the "shared 
variables" will help to do this. Shared variables in Behavior Designer are similar to a 
data map in COMBATXXI HTNs. These variables may need to exist in the parent 
tree if they are not procedurally collected within the child, or referenced tree. While 
behaviors that can gather their own variables are sometimes more convenient, they 
are also more likely to be misused. The custom trees that were built early in this 







Figure A.6: The "DestroyWithinRange" behavior engages the nearest enemy 
within range. 


process did not leverage shared variables, while those that were created later did use 
them. 


A.2 Prefab GameObjects 

Prefabs within Unity are useful for implementing large numbers of similar agents 
or objects. In commercial games, enemy characters are often built as prefabs and 
then "spawned" at various points on a map dynamically. For our purposes, prefabs 
were used to build out units of soldiers, statically placed or dynamically generated 
munitions, and other scene objects such as terrain, buildings, and control measures. 


A.2.1 Agents 

Several agent prefabs were built for this project which helped to increase code reuse. 
Each agent is represented by a large box that is subject to the Unity physics engine, 
and a smaller box for a "nose" which indicates the direction the agent is looking. 
Each agent also has a script attached to it, and the agent scripts are organized into 
a class hierarchy as depicted in Figure A.7. The base class provides the majority 
of functionality, such as health tracking, receiving damage, and removing the agent 
from the scene after it is killed. The subclasses only add or override functionality as 


90 









is needed for their specific functionality, such as an "isMounted" Boolean to track the 
mounted or dismounted status of a soldier. 


AgentState 

Class 

-fc MonoBehaviour 
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^_/ 


VehicleState V 
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Figure A.7: The class hierarchy of scripts attached to soldiers and vehicles. 

The titles of the following sections match the name of the prefab in the Unity project. 


BlueSoldier_ AgentState 

This prefab, the visualization of which is shown in Figure A.8, is a simple model of 
a soldier. It is a box shaped Figure with a health bar floating overhead and a "nose" 
to indicate the direction it is facing. The health bar shrinks and turns progressively 
red as the agent’s health is reduced from 100 to 0. This prefab has the "SoldierState" 
script attached to it, which inherits from the "AgentState" base class. Each soldier 
also has a line Tenderer attached to it which allows for visualization of shots to enemy 
targets. 



Figure A.8: The 11 BlueSoldier_AgentState" prefab 


There is a "BlueDeath" prefab associated with this prefab, so that when the agent 
dies, a partially transparent blue "X" is placed where they died. This is simply for 
reference, and there is no physics-based component to the marker which would inter¬ 
fere with the scene. This can be turned on or off from the SoldierState script attached 
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to all instances of the BlueSoldier_AgentState prefab. The suffix "AgentState" of 
this prefab was added to the prefab that has the SoldierState script attached to it. 


RedSoldier_ AgentState 

The red soldiers in this project are identical to the blue soldiers with the exception 
of their color and their tag. Red soldiers have a tag titled "redTarget", whereas blue 
soldiers have a "blueTarget" tag. Similar to "BlueDeath", there is a "RedDeath" 
that marks the spot of where red soldiers die. 


BlueVic_ AgentState 

This prefab is based on the BlueSoldier_AgentState prefab, and its attached script, 
"VehicleState", inherits from the "AgentState" base class. Only a basic level of 
functionality was programmed into the vehicle prefab in order to allow it to carry 
soldiers around the battlefield. Instances of the vehicle prefab can be re-sized as 
needed to emulate a certain type of vehicle. 


A.2.2 Munitions 

There are two ways that agents can be damaged in this project. The first method 
is by direct fire, which is achieved by the firing agent calling the publicly available 
ReceiveDamage() method from the target agent’s AgentState or appropriate sub class 
script. 

The second method, which is achieved by the munition class, is through explosions. 
Explosions are modeled with the Munition class or one of its subclass scripts, which 
folds all enemy agents within a specified effective killing radius, and determines how 
much damage should be applied to each enemy agent within that radius. This is 
determined by a function which applies maximum damage within some range and 
then linearly dissipates damage the farther the enemy is away from the detonation 
point. Zero damage is applied beyond the effective killing radius. The munition base 
class and its subclasses are depicted in Figure A.9. Each munition-type prefab has a 
script attached to it which fits within this hierarchy. 
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Figure A.9: The class hierarchy of scripts attached to munitions 


Proximity _ IED 

This prefab utilizes the Munition IED class script, which inherits from the Munition 
base class. The IED is represented by a small red box. The small box has an empty 
box around it which is meant to emulate a sensor area of interest in COMBATXXI, 
which can be used to set off an IED. This empty box in Unity is used as a trigger, 
which leverages the OnTriggerEnterQ method within the Unity physics engine. The 
associated script detects any agent entering the box, and if it is an enemy agent, 
it detonates, mimicking a closely-monitored command wire IED. The prefab also 
includes a detonation sound and particle system, which are initiated at the same 
time as the damage function, for debugging purposes. The Proximity_IED prefab is 
depicted in Figure A. 10. 



Figure A.10: A proximity IED which detonates when an enemy agent enters 
the empty box 


60mmMortar 

The second type of munition within this project is intended to emulate IDF. This a 
simple model of IDF; there is no ballistics modeling, and the impact area is symmet¬ 
rical. There are no actual artillery agents, nor is there a fire direction center (FDC). 
Instead, rounds of 60 millimeter mortars are simply spawned and immediately deto¬ 
nated at set intervals at random points within a user-defined box. This prefab was 
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built in conjunction with the custom "call for fire" task, which allows a user to input 
these parameters, and build a call for fire functionality into a behavior tree. 

This prefab, shown in action in Figure A.11, is used in conjunction with the "ID- 
FZone" and "IDFMarker" prefabs. The latter marks the center point of the area 
which is to be targeted. The "IDFZone" is spawned based on the center location 
of the "IDFMarker", and this dictates the space in which the IDF rounds can land. 
Each round’s location is determined by a random draw from a uniform distribution 
in both the length and width of the gray plane. An action shot of this behavior is 
depicted in Figure A. 11. 



Figure A.11: The red agent on the left is calling for fire, using the 60mm- 
Mortar prefab, while blue agents maneuver through the impact zone. 


A.2.3 Scene Elements 

When prototyping behaviors using the sand box pattern, it is useful to have a prefab 
which includes a baseline of items needed to prototype a behavior. When a user cre¬ 
ates a new scene in Unity, it will be empty. One could simply "save as" a scene which 
is already in use, but this may incur unnecessary baggage from that scene. Some¬ 
times it is useful to start from a clean, yet basically functional slate. Similarly, some 
objects, such as buildings and control measures, are largely uniform, and rebuilding 
them each time one is needed is wasteful. This section describes some prefabs that 
were built for this project and were used multiple times. Each of these objects is 
passive, meaning no behaviors were given to them. 
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Control Measures 

Waypoints were implemented as empty, transparent disks which allow agents to use 
the "patrol" task to move along a series of waypoints. These waypoints light up with 
a bright yellow sphere within the scene window when they are actively being used as 
a patrol route. This is a part of the built-in debugging features that Opsive created 
for Behavior Designer. The waypoints were the only control measures implemented 
for this project, but others, such as phase lines, could be implemented using a thin 
box and the OnTriggerEnterQ method. 

Buildings 

Buildings were used for two purposes in this project. First, they are used to obstruct 
an agent’s line of sight. Even if an enemy agent is within detection range, they 
will not be detected if they are on the other side of a building. With this in mind, 
buildings can also be used to roughly model terrain obstructions such as hills or 
ridges. Secondly, buildings were given a tag identifying them as buildings, so that 
behaviors could be built which count the buildings and make a decision based on the 
number of buildings. This is similar to a common scripted COMBATXXI method, 
which returns the number of buildings along a road. 

BasicScene 

The basic scene, depicted in Figure A.12, includes a 15 x 15 terrain, six buildings, 
area lighting, and a camera which provides an overview for the entire scene. 31 In 
order to use the basic scene prefab, start with a new scene, and drag the prefab in. 
Then, remove the original lighting and camera that came with the new scene, and 
reset the origin of the basic scene prefab to the world space origin. Then, users can 
drop agents and control measures onto the scene and they will "snap" to the terrain 
at the appropriate elevation. 

ComplexScene 

The complex scene was built in order to accommodate larger schemes of maneuver, 
and is depicted in Figure A. 13. Otherwise its basic elements are the same as the basic 

31 Size in Unity is arbitrary. However, most Unity applications that map to the real world use a 
convention that one Unity unit equates to one meter. 
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Figure A.12: The basic scene includes only terrain and several buildings. 

scene. The primary addition is the debugging user interface. This UI was built using 
a Unity UI, and is shown in the camera view at run time. It can be turned off by 
deactivating it in the scene hierarchy. 




Debugging GUI 

0SliiwHea/t/, Bars 
®^*P«o. tons 
aPlayS °una s 


f. 


Figure A.13: The complex scene is larger than the basic scene, and includes 
a debugging user interface built into the game window. 


A.3 Scenes for Testing Basic Behaviors 

The functions of the scenes that are included in this project fall into three categories: 
basic behavior testing, COMBATXXI behavior replication, and new behavior proto¬ 
typing for use in COMBATXXI. The basic behavior scenes were used like sand boxes, 
where small increments of development were conducted that would later be included 
in one of the more complex scenes. This method allows users to reduce complexity 
and focus on perfecting basic behaviors without interference from other behaviors. 
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A.3.1 Basic Combat Engagement Functionality 

The scene titled "BasicEngage" was used as a sand box to test various engagement 
behaviors. It has one blue soldier, a cluster of red soldiers, and a building to impede 
line of sight. This basic set up allows a developer to test how an engagement behavior 
will deal with specific cases such as multiple enemies being within range at once, or 
enemies that go from within range to out of range and back in again. After clicking 
the "run" button, a developer can just select either the blue entity or the cluster of 
red entities and move them around. No movement behavior is needed to test the 
engagement functionality, although that could also be tested here. 


A.3.2 Basic IED Functionality 

Often in a combat simulation scenario, an IED is not triggered until later in the 
scenario. It may be more time effective to test the behavior of the IED in a small 
scenario, such as the scene titled "IED_test". As discussed in the munition section, 
munitions should induce less damage to targets that are farther away. An easy way 
to test that this functionality is working is to march a formation directly into an IED 
blast zone. The blue soldiers in this scene have a behavior which first puts them in a 
formation, and then marches them towards the IED. Using the health bars on top of 
each of the agents and noting the marker where any agents may die, it is easy to see 
that the agents that were closer to the blast received more damage. The console and 
inspector windows can be used to confirm this as well. 


A.4 Scenes for Implementing Existing Behaviors 

Several behaviors from existing HTN literature or COMBATXXI tutorials were built 
first in order to gain familiarity with the transferability of HTNs and behavior trees. 


A.4.1 GoToStore 

The classic example that is used to introduce new users to HTNs is the GoToStore be¬ 
havior, discussed in detail in section 2.1.5. An example of this behavior implemented 
as a behavior tree is included in the scene titled, not surprisingly, "goToStore". 
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A.4.2 BOCME 

The Basic Observe, Move, Communicate, and Engage, or BOCME scenario, is used 
to introduce novice COMBATXXI scenario designers to the interfaces needed to build 
new or interpret existing scenarios. 

A.4.3 Bounding Overwatch 

The bounding overwatch scene is similar to the the tutorial by the same name which 
is used to introduce students to more complex HTNs. In this scenario, a squad leader 
recognizes that there is sufficient cover for bounding overwatch. He then starts to 
move the squad down the road alternating in fire teams, using buildings as cover. If 
the user selects the buildings in the scene hierarchy and deactivates them, the squad 
leader will then determine that there is no cover. The squad will then move in a 
squad wedge down the middle of the road. 

A.5 Scenes for Building New Behaviors 

The ultimate goal of this thesis was to build new behaviors in Unity which could then 
be translated to COMBATXXI. The primary scene used in this endeavor is titled 
"platoonSecurityPatrol". When this scene is played, a fire team of three agents patrol 
in a wedge to a vehicle. They load into the vehicle, and the vehicle begins moving to 
a designated drop-off location, at which point the fire team dismounts. The fire team 
then maneuvers in a wedge along a specified route in order to attack a red soldier. 
Meanwhile, this red soldier is calling for fire, resulting in 60mm mortar rounds being 
dropped on the area where the blue fire team is maneuvering. This scene, part of 
which is shown in Figure A. 11, is meant to demonstrate a more complex scheme of 
maneuver which can be built in Unity. 
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APPENDIX B: 

MonoBehaviour Script Lifecycle 


Figure B.l depicts the MonoBehaviour lifecycle, which is the heartbeat of the Unity 
game engine. Most of the lifecycle revolves around the frame update sequence. Frame 
rates vary based on scene complexity and hardware capability. 
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Figure B.l: MonoBehaviour Script Lifecycle from [44] 
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